5.0.0 policy-handler - new PDP API or old PDP API
- in R4 Dublin the policy-engine introduced a totally new API
- policy-handler now has a startup option to either use the new PDP API
or the old PDP API that was created-updated before the end of 2018
- see README.md and README_pdp_api_v0.md for instructions on how to
setup the policy-handler running either with the new PDP API
or the old (pdp_api_v0) PDP API
- this is a massive refactoring that changed almost all the source files,
but kept the old logic when using the old (pdp_api_v0) PDP API
- all the code related to PDP API version is split into two subfolders
= pdp_api/ contains the new PDP API source code
= pdp_api_v0/ contains the old (2018) PDP API source code
= pdp_client.py imports from either pdp_api or pdp_api_v0
= the rest of the code is only affected when it needs to branch
the logic
- logging to policy_handler.log now shows the path of the source file to
allow tracing which PDP API is actually used
- when the new PDP API is used, the policy-update flow is disabled
= passive mode of operation
= no web-socket
= no periodic catch_up
= no policy-filters
= reduced web-API - only a single /policy_latest endpoint is available
/policies_latest returns 404
/catch_up request is accepted, but ignored
- on new PDP API: http /policy_latest returns the new data from the
new PDP API with the following fields added by the policy-handler
to keep other policy related parts intact in R4
(see pdp_api/policy_utils.py)
= "policyName" = policy_id + "." + "policyVersion" + ".xml"
= "policyVersion" = str("metadata"."policy-version")
= "config" - is the renamed "properties" from the new PDP API response
- unit tests are split into two subfolders as well
= main/ for the new PDP API testing
= pdp_api_v0/ for the old (2018) PDP API
- removed the following line from the license text of changed files
ECOMP is a trademark and service mark of AT&T Intellectual Property.
- the new PDP API is expected to be extended and redesigned in R5 El Alto
- on retiring the old PDP API - the intention is to be able to remove
the pdp_api_v0/ subfolder and minimal related cleanup of the code
that imports that as well as the cleanup of the config.py, etc.
Change-Id: Ief9a2ae4541300308caaf97377f4ed051535dbe4
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-1128
diff --git a/LICENSE.txt b/LICENSE.txt
index c142ce1..c6c6e24 100644
--- a/LICENSE.txt
+++ b/LICENSE.txt
@@ -34,7 +34,6 @@
*
* ============LICENSE_END============================================
*
-* ECOMP is a trademark and service mark of AT&T Intellectual Property.
*
*/
diff --git a/README.md b/README.md
index 65c2eba..3266b2f 100644
--- a/README.md
+++ b/README.md
@@ -5,7 +5,7 @@
## web-service for policies to be used by DCAE-Controller
- GET `/policy_latest/<policy_id>` -- get the latest policy from policy-engine that is identified by `policy_id`
-- POST `/policies_latest` -- gets the latest policies that match to the **policy-filter** provided in the body of the request. The policy-filter mimics the body of the /getConfig on policy-engine.
+- POST `/policies_latest` -- *only for the old(pdp_api_v0) PDP API* -- gets the latest policies that match to the **policy-filter** provided in the body of the request. The policy-filter mimics the body of the /getConfig on policy-engine.
sample request - policy-filter
@@ -66,6 +66,8 @@
{
"wservice_port" : 25577,
"consul_url" : "http://consul:8500",
+ "consul_timeout_in_secs" : 60,
+ "pdp_api_version" : null,
"policy_handler" : {
"system" : "policy_handler",
"tls" : {
@@ -84,6 +86,10 @@
- `wservice_port` - port of the policy-hanlder web-service
- `consul_url` - optional url for the consul agent
+- `consul_timeout_in_secs` - optional timeout in seconds to wait for the response from consul agent
+- `pdp_api_version` - optional value for PDP_API_VERSION.
+ - The default PDP API is used when this field is null or absent. The new PDP as of 2019 is the default
+ - To use the old PDP API that was created before the end of 2018, put any value like pdp_api_v0 into this field. Alternatlively, provide a non-empty environment variable $PDP_API_VERSION=pdp_api_v0 on the run of the docker container
- `policy_handler` - local config for policy-handler application
- `system` - general system name of the policy-handler
- `tls` - tls settings for the https clients and server - required to enable tls
@@ -96,6 +102,254 @@
----------
+## discoverable configure from consul-kv for the **new PDP API**
+
+on the start of the policy-handler, it will get the discoverable part of the configure from consul-kv for the key taken from the `local-config.policy_handler.system`
+
+See [README_pdp_api_v0.md](Readme_pdp_api_v0.md) for instructions on how to set up the policy-handler to work with the **old PDP API** that was created not later than **2018**
+
+### using the new PDP API
+
+As of **R4 Dublin** release, the PDP API is totally redesigned. The policy-handler is changed to have a startup option to either using the new PDP API, or the old PDP API (pdp_api_v0).
+
+By **default**, the policy-handler will startup configured to use only the **new PDP API**.
+
+#### service_mode in healthcheck
+
+**R4 Dublin**: when the polcy-handler runs against the default **new PDP API**, the /healthchek response should contain the following values under service_mode element
+
+```json
+{
+ ...
+ "service_mode": {
+ "is_active_mode_of_operation": false,
+ "is_pdp_api_default": true
+ }
+}
+
+```
+
+#### make sure that the start up of the policy-handler is configured to use the **new PDP API**
+
+make sure that both of the following settings are set properly
+
+- make sure that the environment variable `$PDP_API_VERSION` is either **empty** or **not set** at all on the run of the docker container of the policy-handler
+
+- make sure that the `pdp_api_version` param is either `null` or not present in the startup config of the policy-handler at `etc/config.json`
+
+```json
+{
+ "pdp_api_version" : null,
+}
+```
+
+#### point the discovarable config of the policy-handler to point to the **new PDP API**
+
+In short: keep the consul-kv record for he policy-handler as before R4 Dublin.
+
+Here is a sample config from consul-kv. Please replace the {{ ... }} with real setup values
+
+```json
+{
+ ...
+ "policy_engine": {
+ "url": "https://{{ policy_ip_addr }}:{{ policy_ip_port }}",
+ "path_decision": "/decision/v1",
+ "tls_ca_mode": "cert_directory",
+ "timeout_in_secs": 60,
+ "target_entity": "policy_engine",
+ "headers": {
+ "Accept": "application/json",
+ "Content-Type": "application/json",
+ "Authorization": "Basic {{ YOUR_POLICY_ENGINE_AUTHORIZATION }}",
+ "ClientAuth": "Basic {{ YOUR_POLICY_ENGINE_CLIENT_AUTH }}",
+ "Environment": "{{ YOUR_POLICY_ENGINE_ENVIRONMENT }}"
+ }
+ }
+}
+```
+
+----------
+
+## full discoverable configure from consul-kv
+
+```json
+{
+ "policy_handler": {
+ "thread_pool_size": 4,
+ "pool_connections": 20,
+ "policy_retry_count": 5,
+ "policy_retry_sleep": 5,
+ "mode_of_operation": "active",
+ "catch_up": {
+ "interval": 1200
+ },
+ "reconfigure": {
+ "interval": 600
+ },
+ "policy_engine": {
+ "url": "{{ YOUR_POLICY_ENGINE_URL }}",
+ "path_decision": "/decision/v1",
+ "headers": {
+ "Accept": "application/json",
+ "Content-Type": "application/json",
+ "ClientAuth": "Basic {{ YOUR_POLICY_ENGINE_CLIENT_AUTH }}",
+ "Authorization": "Basic {{ YOUR_POLICY_ENGINE_AUTHORIZATION }}",
+ "Environment": "{{ YOUR_POLICY_ENGINE_ENVIRONMENT }}"
+ },
+ "target_entity": "policy_engine",
+ "tls_ca_mode": "cert_directory",
+ "timeout_in_secs": 60
+ },
+ "deploy_handler": {
+ "target_entity": "deployment_handler",
+ "url": "http://deployment_handler:8188",
+ "max_msg_length_mb": 5,
+ "query": {
+ "cfy_tenant_name": "default_tenant"
+ },
+ "tls_ca_mode": "cert_directory",
+ "timeout_in_secs": 60
+ },
+ "service_activator": {
+ "target_entity": "service_activator",
+ "url": "http://service_activator:123",
+ "path_register": "/register",
+ "tls_ca_mode": "cert_directory",
+ "timeout_in_secs": 20,
+ "post_register": {
+ "component_name": "policy_handler",
+ "reconfigure_path": "/reconfigure",
+ "http_protocol": "http"
+ }
+ }
+ }
+}
+```
+
+### field description in yaml format that is equivalent to the actual json structure of the full discoverable config
+
+```yaml
+ policy_handler :
+ # parallelize the getConfig queries to policy-engine on each policy-update notification
+ thread_pool_size : 4
+
+ # parallelize requests to policy-engine and keep them alive
+ pool_connections : 20
+
+ # retry to getConfig from policy-engine on policy-update notification
+ policy_retry_count : 5
+ policy_retry_sleep : 5
+
+ # mode of operation for the policy-handler
+ # either active or passive
+ # in passive mode the policy-hanlder will not listen to
+ # and will not bring the policy-updates from policy-engine
+ mode_of_operation : "active"
+
+ # config of automatic catch_up for resiliency
+ catch_up :
+ # interval in seconds on how often to call automatic catch_up
+ # example: 1200 is 20*60 seconds that is 20 minutes
+ interval : 1200
+
+ # config of periodic reconfigure-rediscover for adaptability
+ reconfigure:
+ # interval in seconds on how often to call automatic reconfigure
+ # example: 600 is 10*60 seconds that is 10 minutes
+ interval : 600
+
+ # PDP (policy-engine) config
+ # These are the url of and the auth for the external system, namely the policy-engine (PDP).
+ # We obtain that info manually from PDP folks at the moment.
+ # In long run we should figure out a way of bringing that info into consul record
+ # related to policy-engine itself.
+ policy_engine :
+ url : "{{ YOUR_POLICY_ENGINE_URL }}"
+ # path to decision on the new PDP API as of 2019
+ path_decision : "/decision/v1"
+ headers :
+ Accept : "application/json"
+ "Content-Type" : "application/json"
+ ClientAuth : "Basic {{ YOUR_POLICY_ENGINE_CLIENT_AUTH }}"
+ Authorization : "Basic {{ YOUR_POLICY_ENGINE_AUTHORIZATION }}"
+ Environment : "{{ YOUR_POLICY_ENGINE_ENVIRONMENT }}"
+ target_entity : "policy_engine"
+ # optional tls_ca_mode specifies where to find the cacert.pem for tls
+ # can be one of these:
+ # "cert_directory" - use the cacert.pem stored locally in cert_directory.
+ # this is the default if cacert.pem file is found
+ #
+ # "os_ca_bundle" - use the public ca_bundle provided by linux system.
+ # this is the default if cacert.pem file not found
+ #
+ # "do_not_verify" - special hack to turn off the verification by cacert and hostname
+ tls_ca_mode : "cert_directory"
+ # optional timeout_in_secs specifies the timeout for the http requests
+ timeout_in_secs: 60
+
+ # deploy_handler config
+ # changed from string "deployment_handler" in 2.3.1 to structure in 2.4.0
+ deploy_handler :
+ # name of deployment-handler service used by policy-handler for logging
+ target_entity : "deployment_handler"
+ # url of the deployment-handler service for policy-handler to direct the policy-updates to
+ # - expecting dns to resolve the name deployment_handler to ip address
+ url : "http://deployment_handler:8188"
+ # limit the size of a single data segment for policy-update messages
+ # from policy-handler to deployment-handler in megabytes
+ max_msg_length_mb : 5
+ query :
+ # optionally specify the tenant name for the cloudify under deployment-handler
+ # if not specified the "default_tenant" is used by the deployment-handler
+ cfy_tenant_name : "default_tenant"
+ # optional tls_ca_mode specifies where to find the cacert.pem or skip tls verification
+ # can be one of these:
+ # "cert_directory" - use the cacert.pem stored locally in cert_directory.
+ # this is the default if cacert.pem file is found
+ #
+ # "os_ca_bundle" - use the public ca_bundle provided by linux system.
+ # this is the default if cacert.pem file not found
+ #
+ # "do_not_verify" - special hack to turn off the verification by cacert and hostname
+ tls_ca_mode : "cert_directory"
+ # optional timeout_in_secs specifies the timeout for the http requests
+ timeout_in_secs: 60
+
+ # optional service_activator config
+ # is used to report the active-passive mode_of_operation of the DCAE-C cluster
+ service_activator :
+ # name of service_activator service used by policy-handler for logging
+ target_entity : "service_activator"
+ # url of the service_activator service for policy-handler to detect the mode-of-operation
+ url : "http://service_activator:123"
+ # path-endpoint to posting the registration to get the mode_of_operation
+ path_register : "/register"
+ # optional tls_ca_mode specifies where to find the cacert.pem or skip tls verification
+ # can be one of these:
+ # "cert_directory" - use the cacert.pem stored locally in cert_directory.
+ # this is the default if cacert.pem file is found
+ #
+ # "os_ca_bundle" - use the public ca_bundle provided by linux system.
+ # this is the default if cacert.pem file not found
+ #
+ # "do_not_verify" - special hack to turn off the verification by cacert and hostname
+ tls_ca_mode : "cert_directory"
+ # optional timeout_in_secs specifies the timeout for the http requests
+ timeout_in_secs : 20
+ # /register request message to post to the service_activator
+ # put anything that service_activator expects for the registration of the policy-handler
+ post_register :
+ # discoverable component name
+ component_name : "policy_handler"
+ # endpoint on policy-handler that will receive the POST on reconfigure event
+ reconfigure_path : "/reconfigure"
+ # protocol for the /reconfigure event
+ http_protocol : "http"
+```
+
+----------
+
## run
in folder `policy_handler`:
diff --git a/README_pdp_api_v0.md b/README_pdp_api_v0.md
new file mode 100644
index 0000000..9fd822a
--- /dev/null
+++ b/README_pdp_api_v0.md
@@ -0,0 +1,265 @@
+# instructions on how to set up the policy-handler to work with the **old PDP API** that was created not later than **2018**
+
+As of **R4 Dublin** release, the PDP API is totally redesigned. The policy-handler is changed to have a startup option to either using the **new PDP API**, or the **old PDP API (pdp_api_v0)**.
+
+By **default**, the policy-handler will startup configured to use only the **new PDP API**.
+
+Follow the below instructions to setup the policy-hanlder for using the **old PDP API** that was created not later than **2018**
+
+## configure the start up of the policy-handler to use the **old PDP API**
+
+there are two options
+
+- **option #1** - provide a non-empty environment variable `$PDP_API_VERSION` on the run of the docker container of the policy-handler like this
+
+```bash
+export PDP_API_VERSION=pdp_api_v0
+docker run ... -e PDP_API_VERSION ...
+```
+
+- **option #2** - if the option#1 is not available, populate the `pdp_api_version` with any not-null value in the startup config of the policy-handler at `etc/config.json`
+
+```json
+{
+ ...
+ "pdp_api_version" : "pdp_api_v0",
+ ...
+}
+```
+
+----------
+
+## point the discovarable config of the policy-handler to point to the **old PDP API**
+
+In short: keep the consul-kv record for he policy-handler as before R4 Dublin.
+
+Here is a sample config from consul-kv. Please replace the {{ ... }} with real setup values
+
+```json
+{
+ ...
+ "policy_engine": {
+ "url": "https://{{ policy_ip_addr }}:{{ policy_ip_port }}",
+ "path_api": "/pdp/api/",
+ "path_notifications": "/pdp/notifications",
+ "tls_ca_mode": "cert_directory",
+ "timeout_in_secs": 60,
+ "tls_wss_ca_mode": "cert_directory",
+ "ws_ping_interval_in_secs": 30,
+ "target_entity": "policy_engine",
+ "headers": {
+ "Accept": "application/json",
+ "Content-Type": "application/json",
+ "Authorization": "Basic {{ YOUR_POLICY_ENGINE_AUTHORIZATION }}",
+ "ClientAuth": "Basic {{ YOUR_POLICY_ENGINE_CLIENT_AUTH }}",
+ "Environment": "{{ YOUR_POLICY_ENGINE_ENVIRONMENT }}"
+ }
+ }
+}
+```
+
+----------
+
+## service_mode in healthcheck
+
+**R4 Dublin**: when the polcy-handler runs against the **old PDP API** that is not default, the /healthchek response should contain the following values under service_mode element
+
+```json
+{
+ ...
+ "service_mode": {
+ "is_active_mode_of_operation": true/false, <<< depends on the mode_of_operation
+ "is_pdp_api_default": false
+ }
+}
+
+```
+
+----------
+
+## full discoverable configure from consul-kv
+
+```json
+{
+ "policy_handler": {
+ "thread_pool_size": 4,
+ "pool_connections": 20,
+ "policy_retry_count": 5,
+ "policy_retry_sleep": 5,
+ "mode_of_operation": "active",
+ "catch_up": {
+ "interval": 1200
+ },
+ "reconfigure": {
+ "interval": 600
+ },
+ "policy_engine": {
+ "url": "{{ YOUR_POLICY_ENGINE_URL }}",
+ "path_notifications": "/pdp/notifications",
+ "path_api": "/pdp/api/",
+ "headers": {
+ "Accept": "application/json",
+ "Content-Type": "application/json",
+ "ClientAuth": "Basic {{ YOUR_POLICY_ENGINE_CLIENT_AUTH }}",
+ "Authorization": "Basic {{ YOUR_POLICY_ENGINE_AUTHORIZATION }}",
+ "Environment": "{{ YOUR_POLICY_ENGINE_ENVIRONMENT }}"
+ },
+ "target_entity": "policy_engine",
+ "tls_ca_mode": "cert_directory",
+ "tls_wss_ca_mode": "cert_directory",
+ "timeout_in_secs": 60,
+ "ws_ping_interval_in_secs": 30
+ },
+ "deploy_handler": {
+ "target_entity": "deployment_handler",
+ "url": "http://deployment_handler:8188",
+ "max_msg_length_mb": 5,
+ "query": {
+ "cfy_tenant_name": "default_tenant"
+ },
+ "tls_ca_mode": "cert_directory",
+ "timeout_in_secs": 60
+ },
+ "service_activator": {
+ "target_entity": "service_activator",
+ "url": "http://service_activator:123",
+ "path_register": "/register",
+ "tls_ca_mode": "cert_directory",
+ "timeout_in_secs": 20,
+ "post_register": {
+ "component_name": "policy_handler",
+ "reconfigure_path": "/reconfigure",
+ "http_protocol": "http"
+ }
+ }
+ }
+}
+```
+
+### field description in yaml format that is equivalent to the actual json structure of the full discoverable config
+
+```yaml
+ policy_handler :
+ # parallelize the getConfig queries to policy-engine on each policy-update notification
+ thread_pool_size : 4
+
+ # parallelize requests to policy-engine and keep them alive
+ pool_connections : 20
+
+ # retry to getConfig from policy-engine on policy-update notification
+ policy_retry_count : 5
+ policy_retry_sleep : 5
+
+ # mode of operation for the policy-handler
+ # either active or passive
+ # in passive mode the policy-hanlder will not listen to
+ # and will not bring the policy-updates from policy-engine
+ mode_of_operation : "active"
+
+ # config of automatic catch_up for resiliency
+ catch_up :
+ # interval in seconds on how often to call automatic catch_up
+ # example: 1200 is 20*60 seconds that is 20 minutes
+ interval : 1200
+
+ # config of periodic reconfigure-rediscover for adaptability
+ reconfigure:
+ # interval in seconds on how often to call automatic reconfigure
+ # example: 600 is 10*60 seconds that is 10 minutes
+ interval : 600
+
+ # PDP (policy-engine) config
+ # These are the url of and the auth for the external system, namely the policy-engine (PDP).
+ # We obtain that info manually from PDP folks at the moment.
+ # In long run we should figure out a way of bringing that info into consul record
+ # related to policy-engine itself.
+ policy_engine :
+ url : "{{ YOUR_POLICY_ENGINE_URL }}"
+ # pathes to the old PDP API created before the end of 2018
+ path_notifications : "/pdp/notifications"
+ path_api : "/pdp/api/"
+ headers :
+ Accept : "application/json"
+ "Content-Type" : "application/json"
+ ClientAuth : "Basic {{ YOUR_POLICY_ENGINE_CLIENT_AUTH }}"
+ Authorization : "Basic {{ YOUR_POLICY_ENGINE_AUTHORIZATION }}"
+ Environment : "{{ YOUR_POLICY_ENGINE_ENVIRONMENT }}"
+ target_entity : "policy_engine"
+ # optional tls_ca_mode specifies where to find the cacert.pem for tls
+ # can be one of these:
+ # "cert_directory" - use the cacert.pem stored locally in cert_directory.
+ # this is the default if cacert.pem file is found
+ #
+ # "os_ca_bundle" - use the public ca_bundle provided by linux system.
+ # this is the default if cacert.pem file not found
+ #
+ # "do_not_verify" - special hack to turn off the verification by cacert and hostname
+ tls_ca_mode : "cert_directory"
+ # optional tls_wss_ca_mode specifies the same for the tls based web-socket
+ tls_wss_ca_mode : "cert_directory"
+ # optional timeout_in_secs specifies the timeout for the http requests
+ timeout_in_secs: 60
+ # optional ws_ping_interval_in_secs specifies the ping interval for the web-socket connection
+ ws_ping_interval_in_secs: 30
+
+ # deploy_handler config
+ # changed from string "deployment_handler" in 2.3.1 to structure in 2.4.0
+ deploy_handler :
+ # name of deployment-handler service used by policy-handler for logging
+ target_entity : "deployment_handler"
+ # url of the deployment-handler service for policy-handler to direct the policy-updates to
+ # - expecting dns to resolve the name deployment_handler to ip address
+ url : "http://deployment_handler:8188"
+ # limit the size of a single data segment for policy-update messages
+ # from policy-handler to deployment-handler in megabytes
+ max_msg_length_mb : 5
+ query :
+ # optionally specify the tenant name for the cloudify under deployment-handler
+ # if not specified the "default_tenant" is used by the deployment-handler
+ cfy_tenant_name : "default_tenant"
+ # optional tls_ca_mode specifies where to find the cacert.pem or skip tls verification
+ # can be one of these:
+ # "cert_directory" - use the cacert.pem stored locally in cert_directory.
+ # this is the default if cacert.pem file is found
+ #
+ # "os_ca_bundle" - use the public ca_bundle provided by linux system.
+ # this is the default if cacert.pem file not found
+ #
+ # "do_not_verify" - special hack to turn off the verification by cacert and hostname
+ tls_ca_mode : "cert_directory"
+ # optional timeout_in_secs specifies the timeout for the http requests
+ timeout_in_secs: 60
+
+ # optional service_activator config
+ # is used to report the active-passive mode_of_operation of the DCAE-C cluster
+ service_activator :
+ # name of service_activator service used by policy-handler for logging
+ target_entity : "service_activator"
+ # url of the service_activator service for policy-handler to detect the mode-of-operation
+ url : "http://service_activator:123"
+ # path-endpoint to posting the registration to get the mode_of_operation
+ path_register : "/register"
+ # optional tls_ca_mode specifies where to find the cacert.pem or skip tls verification
+ # can be one of these:
+ # "cert_directory" - use the cacert.pem stored locally in cert_directory.
+ # this is the default if cacert.pem file is found
+ #
+ # "os_ca_bundle" - use the public ca_bundle provided by linux system.
+ # this is the default if cacert.pem file not found
+ #
+ # "do_not_verify" - special hack to turn off the verification by cacert and hostname
+ tls_ca_mode : "cert_directory"
+ # optional timeout_in_secs specifies the timeout for the http requests
+ timeout_in_secs : 20
+ # /register request message to post to the service_activator
+ # put anything that service_activator expects for the registration of the policy-handler
+ post_register :
+ # discoverable component name
+ component_name : "policy_handler"
+ # endpoint on policy-handler that will receive the POST on reconfigure event
+ reconfigure_path : "/reconfigure"
+ # protocol for the /reconfigure event
+ http_protocol : "http"
+```
+
+----------
diff --git a/etc/config.json b/etc/config.json
index ae5aa51..b6f997b 100644
--- a/etc/config.json
+++ b/etc/config.json
@@ -2,6 +2,7 @@
"wservice_port" : 25577,
"consul_url" : "http://consul:8500",
"consul_timeout_in_secs" : 60,
+ "pdp_api_version" : null,
"policy_handler" : {
"system" : "policy_handler",
"tls" : {
diff --git a/policyhandler/__main__.py b/policyhandler/__main__.py
index 798a2e1..6a71a15 100644
--- a/policyhandler/__main__.py
+++ b/policyhandler/__main__.py
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""
run as server:
@@ -23,22 +22,23 @@
that will invoke this module __main__.py in folder of policyhandler
"""
-import logging
import sys
-from policyhandler import LogWriter
from policyhandler.config import Config
from policyhandler.onap.audit import Audit
-from policyhandler.policy_receiver import PolicyReceiver
-from policyhandler.service_activator import ServiceActivator
-from policyhandler.web_server import PolicyWeb
+from policyhandler.utils import Utils
def run_policy_handler():
"""main run function for policy-handler"""
Config.init_config()
- logger = logging.getLogger("policy_handler")
+ from policyhandler import LogWriter
+ from policyhandler.policy_receiver import PolicyReceiver
+ from policyhandler.service_activator import ServiceActivator
+ from policyhandler.web_server import PolicyWeb
+
+ logger = Utils.get_logger(__file__)
sys.stdout = LogWriter(logger.info)
sys.stderr = LogWriter(logger.error)
diff --git a/policyhandler/config.py b/policyhandler/config.py
index e6e74cc..f8c425a 100644
--- a/policyhandler/config.py
+++ b/policyhandler/config.py
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""read and use the config"""
@@ -25,7 +24,7 @@
import os
from .onap.audit import Audit
-from .policy_utils import Utils
+from .utils import Utils
LOGS_DIR = 'logs'
@@ -40,6 +39,8 @@
'%(threadName)s %(name)s.%(funcName)s: %(message)s'),
datefmt='%Y%m%d_%H%M%S', level=logging.DEBUG)
+_LOGGER = Utils.get_logger(__file__)
+
class Settings(object):
"""settings of module or an application
that is the config filtered by the collection of config-keys.
@@ -127,7 +128,6 @@
class Config(object):
"""main config of the application"""
- _logger = logging.getLogger("policy_handler.config")
CONFIG_FILE_PATH = "etc/config.json"
LOGGER_CONFIG_FILE_PATH = "etc/common_logger.config"
SERVICE_NAME_POLICY_HANDLER = "policy_handler"
@@ -154,9 +154,11 @@
DEFAULT_TIMEOUT_IN_SECS = 60
SERVICE_ACTIVATOR = "service_activator"
MODE_OF_OPERATION = "mode_of_operation"
+ PDP_API_VERSION = "PDP_API_VERSION"
system_name = SERVICE_NAME_POLICY_HANDLER
wservice_port = 25577
+ _pdp_api_version = os.environ.get(PDP_API_VERSION)
consul_url = "http://consul:8500"
consul_timeout_in_secs = DEFAULT_TIMEOUT_IN_SECS
tls_cacert_file = None
@@ -175,7 +177,7 @@
return None
tls_file_path = os.path.join(cert_directory, file_name)
if not os.path.isfile(tls_file_path) or not os.access(tls_file_path, os.R_OK):
- Config._logger.error("invalid %s: %s", tls_name, tls_file_path)
+ _LOGGER.error("invalid %s: %s", tls_name, tls_file_path)
return None
return tls_file_path
@@ -189,19 +191,19 @@
Config.tls_server_ca_chain_file = None
if not (tls_config and isinstance(tls_config, dict)):
- Config._logger.info("no tls in config: %s", json.dumps(tls_config))
+ _LOGGER.info("no tls in config: %s", json.dumps(tls_config))
return
cert_directory = tls_config.get("cert_directory")
if not (cert_directory and isinstance(cert_directory, str)):
- Config._logger.warning("unexpected tls.cert_directory: %r", cert_directory)
+ _LOGGER.warning("unexpected tls.cert_directory: %r", cert_directory)
return
cert_directory = os.path.join(
os.path.dirname(os.path.dirname(os.path.realpath(__file__))), cert_directory)
if not (cert_directory and os.path.isdir(cert_directory)):
- Config._logger.warning("ignoring invalid cert_directory: %s", cert_directory)
+ _LOGGER.warning("ignoring invalid cert_directory: %s", cert_directory)
return
Config.tls_cacert_file = Config._get_tls_file_path(tls_config, cert_directory, "cacert")
@@ -213,16 +215,16 @@
"server_ca_chain")
finally:
- Config._logger.info("tls_cacert_file = %s", Config.tls_cacert_file)
- Config._logger.info("tls_server_cert_file = %s", Config.tls_server_cert_file)
- Config._logger.info("tls_private_key_file = %s", Config.tls_private_key_file)
- Config._logger.info("tls_server_ca_chain_file = %s", Config.tls_server_ca_chain_file)
+ _LOGGER.info("tls_cacert_file = %s", Config.tls_cacert_file)
+ _LOGGER.info("tls_server_cert_file = %s", Config.tls_server_cert_file)
+ _LOGGER.info("tls_private_key_file = %s", Config.tls_private_key_file)
+ _LOGGER.info("tls_server_ca_chain_file = %s", Config.tls_server_ca_chain_file)
@staticmethod
def init_config(file_path=None):
"""read and store the config from config file"""
if Config._local_config.is_loaded():
- Config._logger.info("config already inited: %s", Config._local_config)
+ _LOGGER.info("config already inited: %s", Config._local_config)
return
if not file_path:
@@ -234,10 +236,10 @@
loaded_config = json.load(config_json)
if not loaded_config:
- Config._logger.warning("config not loaded from file: %s", file_path)
+ _LOGGER.warning("config not loaded from file: %s", file_path)
return
- Config._logger.info("config loaded from file: %s", file_path)
+ _LOGGER.info("config loaded from file(%s): %s", file_path, Audit.json_dumps(loaded_config))
logging_config = loaded_config.get("logging")
if logging_config:
logging.config.dictConfig(logging_config)
@@ -249,13 +251,15 @@
if not Config.consul_timeout_in_secs or Config.consul_timeout_in_secs < 1:
Config.consul_timeout_in_secs = Config.DEFAULT_TIMEOUT_IN_SECS
+ Config._pdp_api_version = os.environ.get(
+ Config.PDP_API_VERSION, loaded_config.get(Config.PDP_API_VERSION.lower()))
+
local_config = loaded_config.get(Config.SERVICE_NAME_POLICY_HANDLER, {})
Config.system_name = local_config.get(Config.FIELD_SYSTEM, Config.system_name)
Config._set_tls_config(local_config.get(Config.FIELD_TLS))
Config._local_config.set_config(local_config, auto_commit=True)
- Config._logger.info("config loaded from file(%s): %s", file_path, Config._local_config)
@staticmethod
def discover(audit):
@@ -265,14 +269,14 @@
new_config = DiscoveryClient.get_value(audit, discovery_key)
if not new_config or not isinstance(new_config, dict):
- Config._logger.warning("unexpected config from discovery: %s", new_config)
+ _LOGGER.warning("unexpected config from discovery: %s", new_config)
return
- Config._logger.debug("loaded config from discovery(%s): %s",
- discovery_key, Audit.json_dumps(new_config))
+ _LOGGER.debug("loaded config from discovery(%s): %s",
+ discovery_key, Audit.json_dumps(new_config))
Config.discovered_config.set_config(new_config.get(Config.SERVICE_NAME_POLICY_HANDLER))
- Config._logger.info("config from discovery: %s", Config.discovered_config)
+ _LOGGER.info("config from discovery: %s", Config.discovered_config)
@staticmethod
@@ -302,3 +306,11 @@
def get_requests_kwargs(tls_ca_mode=None):
"""generate kwargs with verify for requests based on the tls_ca_mode"""
return {Config.REQUESTS_VERIFY: Config.get_tls_verify(tls_ca_mode)}
+
+ @staticmethod
+ def is_pdp_api_default(log_status=True):
+ """whether to use the old (2018) or the default pdp API (started in 2019)"""
+ is_default = (Config._pdp_api_version is None)
+ if log_status:
+ _LOGGER.info("_pdp_api_version(%s) default(%s)", Config._pdp_api_version, is_default)
+ return is_default
diff --git a/policyhandler/deploy_handler.py b/policyhandler/deploy_handler.py
index 0ffacba..a127e54 100644
--- a/policyhandler/deploy_handler.py
+++ b/policyhandler/deploy_handler.py
@@ -1,5 +1,5 @@
# ================================================================================
-# Copyright (c) 2017-2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2017-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,12 +14,10 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""send policy-update notification to deployment-handler"""
import json
-import logging
from copy import copy, deepcopy
from threading import Lock
@@ -32,7 +30,9 @@
from .policy_consts import (CATCH_UP, LATEST_POLICIES, POLICIES,
POLICY_FILTER_MATCHES, POLICY_FILTERS,
REMOVED_POLICIES, TARGET_ENTITY)
+from .utils import Utils
+_LOGGER = Utils.get_logger(__file__)
class PolicyUpdateMessage(object):
"""class for messages to deployment-handler on policy-update"""
@@ -143,7 +143,6 @@
class DeployHandler(object):
"""calling the deployment-handler web apis"""
- _logger = logging.getLogger("policy_handler.deploy_handler")
DEFAULT_TARGET_ENTITY = "deployment_handler"
DEFAULT_TIMEOUT_IN_SECS = 60
@@ -202,7 +201,7 @@
tls_ca_mode = config_dh.get(Config.TLS_CA_MODE)
DeployHandler._custom_kwargs = Config.get_requests_kwargs(tls_ca_mode)
- DeployHandler._logger.info(
+ _LOGGER.info(
"dns based routing to %s: url(%s) tls_ca_mode(%s) custom_kwargs(%s)",
DeployHandler._target_entity, DeployHandler._url,
tls_ca_mode, json.dumps(DeployHandler._custom_kwargs))
@@ -221,8 +220,8 @@
DeployHandler._target_entity)
DeployHandler._url_policy = str(DeployHandler._url or "") + '/policy'
- DeployHandler._logger.info("got %s policy url(%s): %s", DeployHandler._target_entity,
- DeployHandler._url_policy, DeployHandler._settings)
+ _LOGGER.info("got %s policy url(%s): %s", DeployHandler._target_entity,
+ DeployHandler._url_policy, DeployHandler._settings)
DeployHandler._settings.commit_change()
DeployHandler._lazy_inited = bool(DeployHandler._url)
@@ -310,12 +309,12 @@
json.dumps(params), timeout_in_secs, json.dumps(custom_kwargs))
log_line = log_action + " " + log_data
- DeployHandler._logger.info(log_line)
+ _LOGGER.info(log_line)
metrics.metrics_start(log_line)
if not DeployHandler._url:
error_msg = "no url found to {0}".format(log_line)
- DeployHandler._logger.error(error_msg)
+ _LOGGER.error(error_msg)
metrics.set_http_status_code(AuditHttpCode.SERVICE_UNAVAILABLE_ERROR.value)
audit.set_http_status_code(AuditHttpCode.SERVICE_UNAVAILABLE_ERROR.value)
metrics.metrics(error_msg)
@@ -331,7 +330,7 @@
else AuditHttpCode.SERVER_INTERNAL_ERROR.value)
error_msg = "failed to {} {}: {} {}".format(
log_action, type(ex).__name__, str(ex), log_data)
- DeployHandler._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
metrics.set_http_status_code(error_code)
audit.set_http_status_code(error_code)
metrics.metrics(error_msg)
@@ -345,10 +344,10 @@
metrics.metrics(log_line)
if res.status_code != requests.codes.ok:
- DeployHandler._logger.error(log_line)
+ _LOGGER.error(log_line)
return
- DeployHandler._logger.info(log_line)
+ _LOGGER.info(log_line)
result = res.json() or {}
DeployHandler._server_instance_changed(result, metrics)
@@ -379,12 +378,12 @@
json.dumps(headers), json.dumps(params), timeout_in_secs, json.dumps(custom_kwargs))
log_line = log_action + " " + log_data
- DeployHandler._logger.info(log_line)
+ _LOGGER.info(log_line)
metrics.metrics_start(log_line)
if not DeployHandler._url:
error_msg = "no url found to {}".format(log_line)
- DeployHandler._logger.error(error_msg)
+ _LOGGER.error(error_msg)
metrics.set_http_status_code(AuditHttpCode.SERVICE_UNAVAILABLE_ERROR.value)
audit.set_http_status_code(AuditHttpCode.SERVICE_UNAVAILABLE_ERROR.value)
metrics.metrics(error_msg)
@@ -400,7 +399,7 @@
else AuditHttpCode.SERVER_INTERNAL_ERROR.value)
error_msg = "failed to {} {}: {} {}".format(
log_action, type(ex).__name__, str(ex), log_data)
- DeployHandler._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
metrics.set_http_status_code(error_code)
audit.set_http_status_code(error_code)
metrics.metrics(error_msg)
@@ -414,7 +413,7 @@
metrics.metrics(log_line)
if res.status_code != requests.codes.ok:
- DeployHandler._logger.error(log_line)
+ _LOGGER.error(log_line)
return None, None
result = res.json() or {}
@@ -424,12 +423,12 @@
policy_filters = result.get(POLICY_FILTERS, {})
if not policies and not policy_filters:
audit.set_http_status_code(AuditHttpCode.DATA_NOT_FOUND_OK.value)
- DeployHandler._logger.warning(audit.warn(
+ _LOGGER.warning(audit.warn(
"found no deployed policies or policy-filters: {}".format(log_line),
error_code=AuditResponseCode.DATA_ERROR))
return policies, policy_filters
- DeployHandler._logger.info(log_line)
+ _LOGGER.info(log_line)
return policies, policy_filters
@staticmethod
@@ -442,6 +441,6 @@
and prev_server_instance_uuid != DeployHandler._server_instance_uuid):
DeployHandler.server_instance_changed = True
- DeployHandler._logger.info(metrics.info(
+ _LOGGER.info(metrics.info(
"deployment_handler_changed: {1} != {0}"
.format(prev_server_instance_uuid, DeployHandler._server_instance_uuid)))
diff --git a/policyhandler/discovery.py b/policyhandler/discovery.py
index 4c5b64e..83f54ac 100644
--- a/policyhandler/discovery.py
+++ b/policyhandler/discovery.py
@@ -1,5 +1,5 @@
# ================================================================================
-# Copyright (c) 2017-2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2017-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,20 +14,20 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""client to talk to consul services and kv"""
import base64
import json
-import logging
import requests
from .config import Config
from .customize import CustomizerUser
from .onap.audit import AuditHttpCode, Metrics
+from .utils import Utils
+_LOGGER = Utils.get_logger(__file__)
class DiscoveryClient(object):
"""talking to consul at Config.consul_url
@@ -51,13 +51,12 @@
CONSUL_ENTITY = "consul"
CONSUL_SERVICE_MASK = "{}/v1/catalog/service/{}"
CONSUL_KV_MASK = "{}/v1/kv/{}"
- _logger = logging.getLogger("policy_handler.discovery")
@staticmethod
def _discover_service(audit, service_name, service_path):
"""find the service record in consul"""
response = requests.get(service_path, timeout=Config.consul_timeout_in_secs)
- DiscoveryClient._logger.info(audit.info("response {} from {}: {}".format(
+ _LOGGER.info(audit.info("response {} from {}: {}".format(
response.status_code, service_path, response.text)))
response.raise_for_status()
@@ -75,7 +74,7 @@
log_line = "get from {} at {}".format(DiscoveryClient.CONSUL_ENTITY, service_path)
- DiscoveryClient._logger.info(metrics.metrics_start(log_line))
+ _LOGGER.info(metrics.metrics_start(log_line))
status_code = None
try:
(status_code,
@@ -86,7 +85,7 @@
else AuditHttpCode.SERVER_INTERNAL_ERROR.value)
error_msg = ("failed {}/{} to {} {}: {}".format(status_code, error_code, log_line,
type(ex).__name__, str(ex)))
- DiscoveryClient._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
metrics.set_http_status_code(error_code)
audit.set_http_status_code(error_code)
metrics.metrics(error_msg)
@@ -95,15 +94,14 @@
if not service_url:
error_code = AuditHttpCode.DATA_ERROR.value
error_msg = "failed {}/{} to {}".format(status_code, error_code, log_line)
- DiscoveryClient._logger.error(audit.error(error_msg))
+ _LOGGER.error(audit.error(error_msg))
metrics.set_http_status_code(error_code)
audit.set_http_status_code(error_code)
metrics.metrics(error_msg)
return None
log_line = "response {} {}".format(status_code, log_line)
- DiscoveryClient._logger.info(audit.info("got service_url: {} after {}"
- .format(service_url, log_line)))
+ _LOGGER.info(audit.info("got service_url: {} after {}".format(service_url, log_line)))
metrics.set_http_status_code(status_code)
audit.set_http_status_code(status_code)
@@ -128,7 +126,7 @@
log_line = "get from {} at {}".format(DiscoveryClient.CONSUL_ENTITY, discovery_url)
- DiscoveryClient._logger.info(metrics.metrics_start(log_line))
+ _LOGGER.info(metrics.metrics_start(log_line))
status_code = None
try:
status_code, value = DiscoveryClient._get_value_from_kv(discovery_url)
@@ -138,7 +136,7 @@
else AuditHttpCode.SERVER_INTERNAL_ERROR.value)
error_msg = ("failed {}/{} to {} {}: {}".format(status_code, error_code, log_line,
type(ex).__name__, str(ex)))
- DiscoveryClient._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
metrics.set_http_status_code(error_code)
audit.set_http_status_code(error_code)
metrics.metrics(error_msg)
diff --git a/policyhandler/onap/audit.py b/policyhandler/onap/audit.py
index db4498a..3c09c16 100644
--- a/policyhandler/onap/audit.py
+++ b/policyhandler/onap/audit.py
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""generic class to keep track of request handling
from receiving it through reponse and log all the activities
@@ -68,6 +67,7 @@
PERMISSION_UNAUTHORIZED_ERROR = 401
PERMISSION_FORBIDDEN_ERROR = 403
RESPONSE_ERROR = 400
+ PAGE_NOT_FOUND_ERROR = 404
SERVER_INTERNAL_ERROR = 500
SERVICE_UNAVAILABLE_ERROR = 503
DATA_ERROR = 1030
@@ -94,7 +94,8 @@
elif http_status_code in [AuditHttpCode.PERMISSION_UNAUTHORIZED_ERROR.value,
AuditHttpCode.PERMISSION_FORBIDDEN_ERROR.value]:
response_code = AuditResponseCode.PERMISSION_ERROR
- elif http_status_code == AuditHttpCode.SERVICE_UNAVAILABLE_ERROR.value:
+ elif http_status_code in [AuditHttpCode.SERVICE_UNAVAILABLE_ERROR.value,
+ AuditHttpCode.PAGE_NOT_FOUND_ERROR.value]:
response_code = AuditResponseCode.AVAILABILITY_ERROR
elif http_status_code == AuditHttpCode.SERVER_INTERNAL_ERROR.value:
response_code = AuditResponseCode.BUSINESS_PROCESS_ERROR
@@ -125,9 +126,9 @@
:kwargs: - put any request related params into kwargs
"""
- _service_name = ""
+ SERVICE_INSTANCE_UUID = str(uuid.uuid4())
+ service_name = ""
_service_version = ""
- _service_instance_uuid = str(uuid.uuid4())
_started = datetime.utcnow()
_key_format = re.compile(r"\W")
_logger_debug = None
@@ -144,15 +145,15 @@
@staticmethod
def init(service_name, config_file_path):
"""init static invariants and loggers"""
- _Audit._service_name = service_name
+ _Audit.service_name = service_name
_Audit._logger_debug = CommonLogger(config_file_path, "debug", \
- instanceUUID=_Audit._service_instance_uuid, serviceName=_Audit._service_name)
+ instanceUUID=_Audit.SERVICE_INSTANCE_UUID, serviceName=_Audit.service_name)
_Audit._logger_error = CommonLogger(config_file_path, "error", \
- instanceUUID=_Audit._service_instance_uuid, serviceName=_Audit._service_name)
+ instanceUUID=_Audit.SERVICE_INSTANCE_UUID, serviceName=_Audit.service_name)
_Audit._logger_metrics = CommonLogger(config_file_path, "metrics", \
- instanceUUID=_Audit._service_instance_uuid, serviceName=_Audit._service_name)
+ instanceUUID=_Audit.SERVICE_INSTANCE_UUID, serviceName=_Audit.service_name)
_Audit._logger_audit = CommonLogger(config_file_path, "audit", \
- instanceUUID=_Audit._service_instance_uuid, serviceName=_Audit._service_name)
+ instanceUUID=_Audit.SERVICE_INSTANCE_UUID, serviceName=_Audit.service_name)
ProcessInfo.init()
try:
_Audit._service_version = subprocess.check_output(
@@ -175,7 +176,7 @@
:req_message: is the request message string for logging
:kwargs: - put any request related params into kwargs
"""
- self.job_name = _Audit._key_format.sub('_', job_name or req_message or _Audit._service_name)
+ self.job_name = _Audit._key_format.sub('_', job_name or req_message or _Audit.service_name)
self.request_id = request_id
self.req_message = req_message or ""
self.kwargs = kwargs or {}
@@ -200,9 +201,9 @@
utcnow = datetime.utcnow()
health = {
"server" : {
- "service_name" : _Audit._service_name,
+ "service_name" : _Audit.service_name,
"service_version" : _Audit._service_version,
- "service_instance_uuid" : _Audit._service_instance_uuid
+ "service_instance_uuid" : _Audit.SERVICE_INSTANCE_UUID
},
"runtime" : {
"started" : str(_Audit._started),
@@ -214,11 +215,12 @@
"process_memory" : ProcessInfo.process_memory()
},
"stats" : _Audit._health.dump(),
- "items" : dict((health_name, health_getter())
- for health_name, health_getter in _Audit._health_checkers.items()),
"soft" : {"python" : _Audit._py_ver, "packages" : _Audit._packages}
}
- self.info("{} health: {}".format(_Audit._service_name,
+ health.update(dict((health_name, health_getter())
+ for health_name, health_getter in _Audit._health_checkers.items())
+ )
+ self.info("{} health: {}".format(_Audit.service_name,
json.dumps(health, sort_keys=True)))
return health
@@ -226,7 +228,7 @@
def process_info(self):
"""get the debug info on all the threads and memory"""
process_info = ProcessInfo.get_all()
- self.info("{} process_info: {}".format(_Audit._service_name, json.dumps(process_info)))
+ self.info("{} process_info: {}".format(_Audit.service_name, json.dumps(process_info)))
return process_info
diff --git a/policyhandler/pdp_api/__init__.py b/policyhandler/pdp_api/__init__.py
new file mode 100644
index 0000000..4d009ed
--- /dev/null
+++ b/policyhandler/pdp_api/__init__.py
@@ -0,0 +1,30 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""2019 http api to policy-engine https://<policy-engine>:<port>/decision/v1/ POST"""
+
+from .policy_matcher import PolicyMatcher
+from .policy_rest import PolicyRest
+from .policy_listener import PolicyListener
+from .policy_updates import PolicyUpdates
+
+def get_pdp_api_info():
+ """info on which version of pdp api is in effect"""
+ return ("folders: PolicyMatcher({}), PolicyRest({}), PolicyListener({}), PolicyUpdates({})"
+ .format(PolicyMatcher.PDP_API_FOLDER, PolicyRest.PDP_API_FOLDER,
+ PolicyListener.PDP_API_FOLDER, PolicyUpdates.PDP_API_FOLDER
+ ))
diff --git a/policyhandler/pdp_api/pdp_consts.py b/policyhandler/pdp_api/pdp_consts.py
new file mode 100644
index 0000000..2337456
--- /dev/null
+++ b/policyhandler/pdp_api/pdp_consts.py
@@ -0,0 +1,35 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+"""contants of PDP"""
+
+# fields from pdp API 2018
+POLICY_NAME = "policyName"
+POLICY_VERSION = "policyVersion"
+POLICY_CONFIG = 'config'
+
+# fields from pdp API 2019
+PDP_POLICIES = 'policies'
+PDP_PROPERTIES = 'properties'
+PDP_METADATA = 'metadata'
+PDP_POLICY_ID = 'policy-id'
+PDP_POLICY_VERSION = 'policy-version'
+
+# req to PDP
+PDP_REQ_ONAP_NAME = "ONAPName" # always "DCAE"
+PDP_REQ_ONAP_COMPONENT = "ONAPComponent"
+PDP_REQ_ONAP_INSTANCE = "ONAPInstance"
+PDP_REQ_RESOURCE = "resource"
diff --git a/policyhandler/pdp_api/policy_listener.py b/policyhandler/pdp_api/policy_listener.py
new file mode 100644
index 0000000..9fa4695
--- /dev/null
+++ b/policyhandler/pdp_api/policy_listener.py
@@ -0,0 +1,55 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""
+policy-listener communicates with policy-engine
+to receive push notifications
+on updates and removal of policies.
+
+on receiving the policy-notifications, the policy-receiver
+passes the notifications to policy-updater
+"""
+
+import os
+
+from ..utils import ToBeImplementedException, Utils
+
+_LOGGER = Utils.get_logger(__file__)
+
+class PolicyListener(object):
+ """listener to PolicyEngine"""
+ PDP_API_FOLDER = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
+
+ def __init__(self, *_):
+ """listener to receive the policy notifications from PolicyEngine"""
+ _LOGGER.info("to_be_implemented")
+ raise ToBeImplementedException()
+
+ def reconfigure(self, _):
+ """configure and reconfigure the listener"""
+ _LOGGER.info("to_be_implemented")
+ raise ToBeImplementedException()
+
+ def run(self):
+ """listen on web-socket and pass the policy notifications to policy-updater"""
+ _LOGGER.info("to_be_implemented")
+ raise ToBeImplementedException()
+
+ def shutdown(self, _):
+ """Shutdown the policy-listener"""
+ _LOGGER.info("to_be_implemented")
+ raise ToBeImplementedException()
diff --git a/policyhandler/pdp_api/policy_matcher.py b/policyhandler/pdp_api/policy_matcher.py
new file mode 100644
index 0000000..57258c3
--- /dev/null
+++ b/policyhandler/pdp_api/policy_matcher.py
@@ -0,0 +1,25 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""policy-matcher matches the policies from deployment-handler to policies from policy-engine"""
+
+import os
+
+
+class PolicyMatcher(object):
+ """policy-matcher - static class"""
+ PDP_API_FOLDER = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
diff --git a/policyhandler/pdp_api/policy_rest.py b/policyhandler/pdp_api/policy_rest.py
new file mode 100644
index 0000000..14d9296
--- /dev/null
+++ b/policyhandler/pdp_api/policy_rest.py
@@ -0,0 +1,215 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""policy-client communicates with policy-engine thru REST API"""
+
+import copy
+import json
+import os
+import urllib.parse
+from threading import Lock
+
+import requests
+
+from ..config import Config, Settings
+from ..onap.audit import (REQUEST_X_ECOMP_REQUESTID, AuditHttpCode,
+ AuditResponseCode, Metrics)
+from ..utils import Utils
+from .pdp_consts import PDP_POLICIES
+from .policy_utils import PolicyUtils
+
+_LOGGER = Utils.get_logger(__file__)
+
+class PolicyRest(object):
+ """using the http API to policy-engine"""
+ PDP_API_FOLDER = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
+ _lazy_inited = False
+ DEFAULT_TIMEOUT_IN_SECS = 60
+
+ _lock = Lock()
+ _settings = Settings(Config.FIELD_POLICY_ENGINE, Config.POOL_CONNECTIONS)
+
+ _target_entity = None
+ _requests_session = None
+ _url = None
+ _url_pdp_decision = None
+ _headers = None
+ _custom_kwargs = {}
+ _timeout_in_secs = DEFAULT_TIMEOUT_IN_SECS
+
+ @staticmethod
+ def _init():
+ """init static config"""
+ PolicyRest._custom_kwargs = {}
+ tls_ca_mode = None
+
+ if not PolicyRest._requests_session:
+ PolicyRest._requests_session = requests.Session()
+
+ changed, pool_size = PolicyRest._settings.get_by_key(Config.POOL_CONNECTIONS, 20)
+ if changed:
+ PolicyRest._requests_session.mount(
+ 'https://', requests.adapters.HTTPAdapter(pool_connections=pool_size,
+ pool_maxsize=pool_size))
+ PolicyRest._requests_session.mount(
+ 'http://', requests.adapters.HTTPAdapter(pool_connections=pool_size,
+ pool_maxsize=pool_size))
+
+ _, config = PolicyRest._settings.get_by_key(Config.FIELD_POLICY_ENGINE)
+ if config:
+ PolicyRest._url = config.get("url")
+ if PolicyRest._url:
+ PolicyRest._url_pdp_decision = urllib.parse.urljoin(
+ PolicyRest._url, config.get("path_decision", "/decision/v1/"))
+ PolicyRest._headers = config.get("headers", {})
+ PolicyRest._target_entity = config.get("target_entity", Config.FIELD_POLICY_ENGINE)
+
+ tls_ca_mode = config.get(Config.TLS_CA_MODE)
+ PolicyRest._custom_kwargs = Config.get_requests_kwargs(tls_ca_mode)
+ PolicyRest._timeout_in_secs = config.get(Config.TIMEOUT_IN_SECS)
+ if not PolicyRest._timeout_in_secs or PolicyRest._timeout_in_secs < 1:
+ PolicyRest._timeout_in_secs = PolicyRest.DEFAULT_TIMEOUT_IN_SECS
+
+ _LOGGER.info(
+ "PDP(%s) url(%s) headers(%s) tls_ca_mode(%s) timeout_in_secs(%s) custom_kwargs(%s): %s",
+ PolicyRest._target_entity, PolicyRest._url_pdp_decision,
+ Metrics.json_dumps(PolicyRest._headers), tls_ca_mode,
+ PolicyRest._timeout_in_secs, json.dumps(PolicyRest._custom_kwargs),
+ PolicyRest._settings)
+
+ PolicyRest._settings.commit_change()
+ PolicyRest._lazy_inited = True
+
+ @staticmethod
+ def reconfigure():
+ """reconfigure"""
+ with PolicyRest._lock:
+ PolicyRest._settings.set_config(Config.discovered_config)
+ if not PolicyRest._settings.is_changed():
+ PolicyRest._settings.commit_change()
+ return False
+
+ PolicyRest._lazy_inited = False
+ PolicyRest._init()
+ return True
+
+ @staticmethod
+ def _lazy_init():
+ """init static config"""
+ if PolicyRest._lazy_inited:
+ return
+
+ with PolicyRest._lock:
+ if PolicyRest._lazy_inited:
+ return
+
+ PolicyRest._settings.set_config(Config.discovered_config)
+ PolicyRest._init()
+
+ @staticmethod
+ def _pdp_get_decision(audit, pdp_req):
+ """Communication with the policy-engine"""
+ if not PolicyRest._url:
+ _LOGGER.error(
+ audit.error("no url for PDP", error_code=AuditResponseCode.AVAILABILITY_ERROR))
+ audit.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
+ return None
+
+ with PolicyRest._lock:
+ session = PolicyRest._requests_session
+ target_entity = PolicyRest._target_entity
+ url = PolicyRest._url_pdp_decision
+ timeout_in_secs = PolicyRest._timeout_in_secs
+ headers = copy.deepcopy(PolicyRest._headers)
+ custom_kwargs = copy.deepcopy(PolicyRest._custom_kwargs)
+
+ metrics = Metrics(aud_parent=audit, targetEntity=target_entity, targetServiceName=url)
+
+ headers[REQUEST_X_ECOMP_REQUESTID] = metrics.request_id
+
+ log_action = "post to {} at {}".format(target_entity, url)
+ log_data = "msg={} headers={}, custom_kwargs({}) timeout_in_secs({})".format(
+ json.dumps(pdp_req), Metrics.json_dumps(headers), json.dumps(custom_kwargs),
+ timeout_in_secs)
+ log_line = log_action + " " + log_data
+
+ _LOGGER.info(metrics.metrics_start(log_line))
+
+ res = None
+ try:
+ res = session.post(url, json=pdp_req, headers=headers, timeout=timeout_in_secs,
+ **custom_kwargs)
+ except Exception as ex:
+ error_code = (AuditHttpCode.SERVICE_UNAVAILABLE_ERROR.value
+ if isinstance(ex, requests.exceptions.RequestException)
+ else AuditHttpCode.SERVER_INTERNAL_ERROR.value)
+ error_msg = ("failed {}: {} to {}".format(type(ex).__name__, str(ex), log_line))
+
+ _LOGGER.exception(error_msg)
+ metrics.set_http_status_code(error_code)
+ audit.set_http_status_code(error_code)
+ metrics.metrics(error_msg)
+ return None
+
+ log_line = "response {} from {}: text={} headers={}".format(
+ res.status_code, log_line, res.text,
+ Metrics.json_dumps(dict(res.request.headers.items())))
+
+ _LOGGER.info(log_line)
+ metrics.set_http_status_code(res.status_code)
+ audit.set_http_status_code(res.status_code)
+ metrics.metrics(log_line)
+
+ policy_bodies = None
+ if res.status_code == requests.codes.ok:
+ policy_bodies = res.json().get(PDP_POLICIES)
+
+ return policy_bodies
+
+ @staticmethod
+ def get_latest_policy(aud_policy_id):
+ """safely try retrieving the latest policy for the policy_id from the policy-engine"""
+ audit, policy_id, _, _ = aud_policy_id
+ try:
+ PolicyRest._lazy_init()
+
+ pdp_req = PolicyUtils.gen_req_to_pdp(policy_id)
+ policy_bodies = PolicyRest._pdp_get_decision(audit, pdp_req)
+
+ log_line = "looking for policy_id({}) in policy_bodies: {}".format(
+ policy_id, json.dumps(policy_bodies))
+ _LOGGER.info(log_line)
+
+ latest_policy = None
+ if policy_bodies and policy_id in policy_bodies:
+ latest_policy = PolicyUtils.convert_to_policy(policy_bodies[policy_id])
+
+ if not PolicyUtils.validate_policy(latest_policy):
+ audit.set_http_status_code(AuditHttpCode.DATA_NOT_FOUND_OK.value)
+ _LOGGER.error(audit.error(
+ "received invalid policy from PDP: {}".format(json.dumps(latest_policy)),
+ error_code=AuditResponseCode.DATA_ERROR))
+
+ return latest_policy
+ except Exception as ex:
+ error_msg = ("{}: get_latest_policy({}) crash {}: {}"
+ .format(audit.request_id, policy_id, type(ex).__name__, str(ex)))
+
+ _LOGGER.exception(error_msg)
+ audit.fatal(error_msg, error_code=AuditResponseCode.BUSINESS_PROCESS_ERROR)
+ audit.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
+ return None
diff --git a/policyhandler/pdp_api/policy_updates.py b/policyhandler/pdp_api/policy_updates.py
new file mode 100644
index 0000000..eb3c3d1
--- /dev/null
+++ b/policyhandler/pdp_api/policy_updates.py
@@ -0,0 +1,49 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""policy-updates accumulates the policy-update notifications from PDP"""
+
+import os
+
+from ..utils import Utils, ToBeImplementedException
+
+
+_LOGGER = Utils.get_logger(__file__)
+
+class PolicyUpdates(object):
+ """Keep and consolidate the policy-updates (audit, policies_updated, policies_removed)"""
+ PDP_API_FOLDER = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
+
+ def __init__(self):
+ """init and reset"""
+
+ def reset(self):
+ """resets the state"""
+ self.__init__()
+
+ def pop_policy_updates(self):
+ """
+ Returns the consolidated (audit, policies_updated, policies_removed)
+ and resets the state
+ """
+ _LOGGER.info("to_be_implemented")
+ return None, None, None
+
+ def push_policy_updates(self, *_):
+ """consolidate the new policies_updated, policies_removed to existing ones"""
+ _LOGGER.info("to_be_implemented")
+ raise ToBeImplementedException()
diff --git a/policyhandler/pdp_api/policy_utils.py b/policyhandler/pdp_api/policy_utils.py
new file mode 100644
index 0000000..1d06d14
--- /dev/null
+++ b/policyhandler/pdp_api/policy_utils.py
@@ -0,0 +1,123 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""utils for policy usage and conversions"""
+
+from ..onap.audit import Audit
+from ..policy_consts import POLICY_BODY, POLICY_ID
+from .pdp_consts import (PDP_METADATA, PDP_POLICY_ID,
+ PDP_POLICY_VERSION, PDP_PROPERTIES,
+ PDP_REQ_ONAP_COMPONENT, PDP_REQ_ONAP_INSTANCE,
+ PDP_REQ_ONAP_NAME, PDP_REQ_RESOURCE, POLICY_CONFIG,
+ POLICY_NAME, POLICY_VERSION)
+
+
+class PolicyUtils(object):
+ """policy-client utils"""
+
+ @staticmethod
+ def gen_req_to_pdp(policy_id):
+ """request to get a single policy from pdp by policy_id"""
+ return {
+ PDP_REQ_ONAP_NAME: "DCAE",
+ PDP_REQ_ONAP_COMPONENT: Audit.service_name,
+ PDP_REQ_ONAP_INSTANCE: Audit.SERVICE_INSTANCE_UUID,
+ "action": "configure",
+ PDP_REQ_RESOURCE: {PDP_POLICY_ID: [policy_id]}
+ }
+
+ @staticmethod
+ def convert_to_policy(policy_body):
+ """
+ set policy id, name, version, config=properties and
+ wrap policy_body received from policy-engine with policy_id
+
+ input:
+ {
+ "type": "onap.policies.monitoring.cdap.tca.hi.lo.app",
+ "version": "1.0.0",
+ "metadata": {
+ "policy-id": "onap.scaleout.tca",
+ "policy-version": 1,
+ "description": "The scaleout policy for vDNS"
+ },
+ "properties": {
+ "tca_policy": {
+ "domain": "measurementsForVfScaling",
+ "metricsPerEventName": [
+ {
+ "eventName": "vLoadBalancer",
+ "controlLoopSchemaType": "VNF",
+ "policyScope": "type=configuration"
+ }
+ ]
+ }
+ }
+ }
+
+ output:
+ {
+ "policy_id": "onap.scaleout.tca",
+ "policy_body": {
+ "policyName": "onap.scaleout.tca.1.xml",
+ "policyVersion": 1,
+ "type": "onap.policies.monitoring.cdap.tca.hi.lo.app",
+ "version": "1.0.0",
+ "metadata": {
+ "policy-id": "onap.scaleout.tca",
+ "policy-version": 1,
+ "description": "The scaleout policy for vDNS"
+ },
+ "config": {
+ "tca_policy": {
+ "domain": "measurementsForVfScaling",
+ "metricsPerEventName": [
+ {
+ "eventName": "vLoadBalancer",
+ "controlLoopSchemaType": "VNF",
+ "policyScope": "type=configuration"
+ }
+ ]
+ }
+ }
+ }
+ }
+ """
+ if not policy_body or not policy_body.get(PDP_PROPERTIES):
+ return None
+
+ pdp_metadata = policy_body.get(PDP_METADATA, {})
+ policy_id = pdp_metadata.get(PDP_POLICY_ID)
+ policy_version = pdp_metadata.get(PDP_POLICY_VERSION)
+ if not policy_id or not policy_version:
+ return None
+
+ policy_body[POLICY_NAME] = "{}.{}.xml".format(policy_id, policy_version)
+ policy_body[POLICY_VERSION] = str(policy_version)
+ policy_body[POLICY_CONFIG] = policy_body[PDP_PROPERTIES]
+ del policy_body[PDP_PROPERTIES]
+
+ return {POLICY_ID:policy_id, POLICY_BODY:policy_body}
+
+ @staticmethod
+ def validate_policy(policy):
+ """validate have non-empty config in policy"""
+ if not policy:
+ return False
+
+ policy_body = policy.get(POLICY_BODY)
+ return bool(policy_body and policy_body.get(POLICY_CONFIG))
diff --git a/policyhandler/pdp_api_v0/__init__.py b/policyhandler/pdp_api_v0/__init__.py
new file mode 100644
index 0000000..0196508
--- /dev/null
+++ b/policyhandler/pdp_api_v0/__init__.py
@@ -0,0 +1,30 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""<=2018 http api to policy-engine /getConfig that is going to be replaced in 2019"""
+
+from .policy_matcher import PolicyMatcher
+from .policy_rest import PolicyRest
+from .policy_listener import PolicyListener
+from .policy_updates import PolicyUpdates
+
+def get_pdp_api_info():
+ """info on which version of pdp api is in effect"""
+ return ("folders: PolicyMatcher({}), PolicyRest({}), PolicyListener({}), PolicyUpdates({})"
+ .format(PolicyMatcher.PDP_API_FOLDER, PolicyRest.PDP_API_FOLDER,
+ PolicyListener.PDP_API_FOLDER, PolicyUpdates.PDP_API_FOLDER
+ ))
diff --git a/policyhandler/pdp_api_v0/pdp_consts.py b/policyhandler/pdp_api_v0/pdp_consts.py
new file mode 100644
index 0000000..d1c0b44
--- /dev/null
+++ b/policyhandler/pdp_api_v0/pdp_consts.py
@@ -0,0 +1,23 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""contants of PDP"""
+
+POLICY_VERSION = "policyVersion"
+POLICY_NAME = "policyName"
+POLICY_CONFIG = 'config'
+MATCHING_CONDITIONS = "matchingConditions"
diff --git a/policyhandler/pdp_api_v0/policy_listener.py b/policyhandler/pdp_api_v0/policy_listener.py
new file mode 100644
index 0000000..67e4c49
--- /dev/null
+++ b/policyhandler/pdp_api_v0/policy_listener.py
@@ -0,0 +1,309 @@
+# ================================================================================
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""
+policy-listener communicates with policy-engine
+thru web-socket to receive push notifications
+on updates and removal of policies.
+
+on receiving the policy-notifications, the policy-receiver
+passes the notifications to policy-updater
+"""
+
+import copy
+import json
+import os
+import ssl
+import time
+import urllib.parse
+from datetime import datetime
+from threading import Lock, Thread
+
+import websocket
+
+from ..config import Config, Settings
+from ..onap.audit import Audit
+from ..utils import Utils
+from .pdp_consts import MATCHING_CONDITIONS, POLICY_NAME, POLICY_VERSION
+
+LOADED_POLICIES = 'loadedPolicies'
+REMOVED_POLICIES = 'removedPolicies'
+POLICY_VER = 'versionNo'
+POLICY_MATCHES = 'matches'
+
+_LOGGER = Utils.get_logger(__file__)
+
+class PolicyListener(Thread):
+ """web-socket to PolicyEngine"""
+ PDP_API_FOLDER = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
+ WS_STARTED = "started"
+ WS_START_COUNT = "start_count"
+ WS_CLOSE_COUNT = "close_count"
+ WS_ERROR_COUNT = "error_count"
+ WS_PONG_COUNT = "pong_count"
+ WS_MESSAGE_COUNT = "message_count"
+ WS_MESSAGE_TIMESTAMP = "message_timestamp"
+ WS_STATUS = "status"
+ WS_PING_INTERVAL_DEFAULT = 30
+ WEB_SOCKET_HEALTH = "web_socket_health"
+
+ def __init__(self, audit, policy_updater):
+ """web-socket inside the thread to receive policy notifications from PolicyEngine"""
+ Thread.__init__(self, name="policy_receiver", daemon=True)
+
+ self._policy_updater = policy_updater
+ self._lock = Lock()
+ self._keep_running = True
+ self._settings = Settings(Config.FIELD_POLICY_ENGINE)
+
+ self._sleep_before_restarting = 5
+ self._web_socket_url = None
+ self._web_socket_sslopt = None
+ self._tls_wss_ca_mode = None
+ self._web_socket = None
+ self._ws_ping_interval_in_secs = PolicyListener.WS_PING_INTERVAL_DEFAULT
+ self._web_socket_health = {
+ PolicyListener.WS_START_COUNT: 0,
+ PolicyListener.WS_CLOSE_COUNT: 0,
+ PolicyListener.WS_ERROR_COUNT: 0,
+ PolicyListener.WS_PONG_COUNT: 0,
+ PolicyListener.WS_MESSAGE_COUNT: 0,
+ PolicyListener.WS_STATUS: "created"
+ }
+
+ Audit.register_item_health(PolicyListener.WEB_SOCKET_HEALTH, self._get_health)
+ self.reconfigure(audit)
+
+ def reconfigure(self, audit):
+ """configure and reconfigure the web-socket"""
+ with self._lock:
+ _LOGGER.info(audit.info("web_socket_health {}".format(
+ json.dumps(self._get_health(), sort_keys=True))))
+ self._sleep_before_restarting = 5
+ self._settings.set_config(Config.discovered_config)
+ changed, config = self._settings.get_by_key(Config.FIELD_POLICY_ENGINE)
+
+ if not changed:
+ self._settings.commit_change()
+ return False
+
+ prev_web_socket_url = self._web_socket_url
+ prev_web_socket_sslopt = self._web_socket_sslopt
+ prev_ws_ping_interval_in_secs = self._ws_ping_interval_in_secs
+
+ self._web_socket_sslopt = None
+
+ resturl = urllib.parse.urljoin(config.get("url", "").lower().rstrip("/") + "/",
+ config.get("path_notifications", "/pdp/notifications"))
+
+ self._tls_wss_ca_mode = config.get(Config.TLS_WSS_CA_MODE)
+
+ self._ws_ping_interval_in_secs = config.get(Config.WS_PING_INTERVAL_IN_SECS)
+ if not self._ws_ping_interval_in_secs or self._ws_ping_interval_in_secs < 60:
+ self._ws_ping_interval_in_secs = PolicyListener.WS_PING_INTERVAL_DEFAULT
+
+ if resturl.startswith("https:"):
+ self._web_socket_url = resturl.replace("https:", "wss:")
+
+ verify = Config.get_tls_verify(self._tls_wss_ca_mode)
+ if verify is False:
+ self._web_socket_sslopt = {'cert_reqs': ssl.CERT_NONE}
+ elif verify is True:
+ pass
+ else:
+ self._web_socket_sslopt = {'ca_certs': verify}
+
+ else:
+ self._web_socket_url = resturl.replace("http:", "ws:")
+
+ log_changed = (
+ "changed web_socket_url(%s) or tls_wss_ca_mode(%s)"
+ " or ws_ping_interval_in_secs(%s): %s" %
+ (self._web_socket_url, self._tls_wss_ca_mode, self._ws_ping_interval_in_secs,
+ self._settings))
+ if (self._web_socket_url == prev_web_socket_url
+ and Utils.are_the_same(prev_web_socket_sslopt, self._web_socket_sslopt)
+ and prev_ws_ping_interval_in_secs == self._ws_ping_interval_in_secs):
+ _LOGGER.info(audit.info("not {}".format(log_changed)))
+ self._settings.commit_change()
+ return False
+
+ _LOGGER.info(audit.info(log_changed))
+ self._settings.commit_change()
+
+ self._stop_notifications()
+ return True
+
+ def run(self):
+ """listen on web-socket and pass the policy notifications to policy-updater"""
+ _LOGGER.info("starting policy_receiver...")
+ websocket.enableTrace(True)
+ restarting = False
+ while True:
+ if not self._get_keep_running():
+ break
+
+ self._stop_notifications()
+
+ if restarting:
+ with self._lock:
+ sleep_before_restarting = self._sleep_before_restarting
+ _LOGGER.info(
+ "going to sleep for %s secs before restarting policy-notifications",
+ sleep_before_restarting)
+
+ time.sleep(sleep_before_restarting)
+ if not self._get_keep_running():
+ break
+
+ with self._lock:
+ web_socket_url = self._web_socket_url
+ sslopt = copy.deepcopy(self._web_socket_sslopt)
+ tls_wss_ca_mode = self._tls_wss_ca_mode
+ ws_ping_interval_in_secs = self._ws_ping_interval_in_secs
+
+ _LOGGER.info(
+ "connecting to policy-notifications at %s with sslopt(%s) tls_wss_ca_mode(%s)"
+ " ws_ping_interval_in_secs(%s)",
+ web_socket_url, json.dumps(sslopt), tls_wss_ca_mode, ws_ping_interval_in_secs)
+
+ self._web_socket = websocket.WebSocketApp(
+ web_socket_url,
+ on_open=self._on_ws_open,
+ on_message=self._on_pdp_message,
+ on_close=self._on_ws_close,
+ on_error=self._on_ws_error,
+ on_pong=self._on_ws_pong
+ )
+
+ _LOGGER.info("waiting for policy-notifications...")
+ self._web_socket.run_forever(sslopt=sslopt, ping_interval=ws_ping_interval_in_secs)
+ restarting = True
+
+ Audit.register_item_health(PolicyListener.WEB_SOCKET_HEALTH)
+ _LOGGER.info("exit policy-receiver")
+
+ def _get_keep_running(self):
+ """thread-safe check whether to continue running"""
+ with self._lock:
+ keep_running = self._keep_running
+ return keep_running
+
+ def _stop_notifications(self):
+ """close the web-socket == stops the notification service if running."""
+ with self._lock:
+ if self._web_socket and self._web_socket.sock and self._web_socket.sock.connected:
+ self._web_socket.close()
+ _LOGGER.info("stopped receiving notifications from PDP")
+
+ def _on_pdp_message(self, *args):
+ """received the notification from PDP"""
+ self._web_socket_health[PolicyListener.WS_MESSAGE_COUNT] += 1
+ self._web_socket_health[PolicyListener.WS_MESSAGE_TIMESTAMP] = str(datetime.utcnow())
+ try:
+ message = args and args[-1]
+ _LOGGER.info("Received notification message: %s", message)
+ _LOGGER.info("web_socket_health %s", json.dumps(self._get_health(), sort_keys=True))
+ if not message:
+ return
+ message = json.loads(message)
+
+ if not message or not isinstance(message, dict):
+ _LOGGER.warning("unexpected message from PDP: %s", json.dumps(message))
+ return
+
+ policies_updated = [
+ {POLICY_NAME: policy.get(POLICY_NAME),
+ POLICY_VERSION: policy.get(POLICY_VER),
+ MATCHING_CONDITIONS: policy.get(POLICY_MATCHES, {})}
+ for policy in message.get(LOADED_POLICIES, [])
+ ]
+
+ policies_removed = [
+ {POLICY_NAME: removed_policy.get(POLICY_NAME),
+ POLICY_VERSION: removed_policy.get(POLICY_VER)}
+ for removed_policy in message.get(REMOVED_POLICIES, [])
+ ]
+
+ if not policies_updated and not policies_removed:
+ _LOGGER.info("no policy updated or removed")
+ return
+
+ self._policy_updater.policy_update(policies_updated, policies_removed)
+ except Exception as ex:
+ error_msg = "crash {} {} at {}: {}".format(type(ex).__name__, str(ex),
+ "on_pdp_message", json.dumps(message))
+
+ _LOGGER.exception(error_msg)
+
+ def _on_ws_error(self, error):
+ """report an error"""
+ _LOGGER.exception("policy-notification error %s", str(error))
+ self._sleep_before_restarting = 60 if isinstance(error, ssl.SSLError) else 5
+
+ self._web_socket_health[PolicyListener.WS_STATUS] = "error"
+ self._web_socket_health[PolicyListener.WS_ERROR_COUNT] += 1
+ self._web_socket_health["last_error"] = {
+ "error": str(error), "timestamp": str(datetime.utcnow())
+ }
+ _LOGGER.info("web_socket_health %s", json.dumps(self._get_health(), sort_keys=True))
+
+ def _on_ws_close(self, code, reason):
+ """restart web-socket on close"""
+ self._web_socket_health["last_closed"] = str(datetime.utcnow())
+ self._web_socket_health[PolicyListener.WS_STATUS] = "closed"
+ self._web_socket_health[PolicyListener.WS_CLOSE_COUNT] += 1
+ _LOGGER.info(
+ "lost connection(%s, %s) to PDP web_socket_health %s",
+ code, reason, json.dumps(self._get_health(), sort_keys=True))
+
+ def _on_ws_open(self):
+ """started web-socket"""
+ self._web_socket_health[PolicyListener.WS_STATUS] = PolicyListener.WS_STARTED
+ self._web_socket_health[PolicyListener.WS_START_COUNT] += 1
+ self._web_socket_health[PolicyListener.WS_STARTED] = datetime.utcnow()
+ _LOGGER.info("opened connection to PDP web_socket_health %s",
+ json.dumps(self._get_health(), sort_keys=True))
+
+ def _on_ws_pong(self, pong):
+ """pong = response to pinging the server of the web-socket"""
+ self._web_socket_health[PolicyListener.WS_PONG_COUNT] += 1
+ _LOGGER.info(
+ "pong(%s) from connection to PDP web_socket_health %s",
+ pong, json.dumps(self._get_health(), sort_keys=True))
+
+ def _get_health(self):
+ """returns the healthcheck of the web-socket as json"""
+ web_socket_health = copy.deepcopy(self._web_socket_health)
+ web_socket_health[Config.WS_PING_INTERVAL_IN_SECS] = self._ws_ping_interval_in_secs
+ started = web_socket_health.get(PolicyListener.WS_STARTED)
+ if started:
+ web_socket_health[PolicyListener.WS_STARTED] = str(started)
+ web_socket_health["uptime"] = str(datetime.utcnow() - started)
+ return web_socket_health
+
+
+ def shutdown(self, audit):
+ """Shutdown the policy-listener"""
+ _LOGGER.info(audit.info("shutdown policy-listener"))
+ with self._lock:
+ self._keep_running = False
+
+ self._stop_notifications()
+
+ if self.is_alive():
+ self.join()
diff --git a/policyhandler/policy_matcher.py b/policyhandler/pdp_api_v0/policy_matcher.py
similarity index 89%
rename from policyhandler/policy_matcher.py
rename to policyhandler/pdp_api_v0/policy_matcher.py
index d0786ba..357af49 100644
--- a/policyhandler/policy_matcher.py
+++ b/policyhandler/pdp_api_v0/policy_matcher.py
@@ -1,5 +1,5 @@
# ================================================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,27 +14,28 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""policy-matcher matches the policies from deployment-handler to policies from policy-engine"""
import json
-import logging
+import os
import re
-from .deploy_handler import DeployHandler, PolicyUpdateMessage
-from .onap.audit import AuditHttpCode, AuditResponseCode
-from .policy_consts import (ERRORED_POLICIES, LATEST_POLICIES,
- MATCHING_CONDITIONS, POLICY_BODY, POLICY_FILTER,
- POLICY_NAME, POLICY_VERSION, POLICY_VERSIONS)
+from ..deploy_handler import DeployHandler, PolicyUpdateMessage
+from ..onap.audit import AuditHttpCode, AuditResponseCode
+from ..policy_consts import (ERRORED_POLICIES, LATEST_POLICIES, POLICY_BODY,
+ POLICY_FILTER, POLICY_VERSIONS)
+from ..utils import RegexCoarser, Utils
+from .pdp_consts import MATCHING_CONDITIONS, POLICY_NAME, POLICY_VERSION
from .policy_rest import PolicyRest
-from .policy_utils import RegexCoarser
+_LOGGER = Utils.get_logger(__file__)
+
class PolicyMatcher(object):
"""policy-matcher - static class"""
- _logger = logging.getLogger("policy_handler.policy_matcher")
PENDING_UPDATE = "pending_update"
+ PDP_API_FOLDER = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
@staticmethod
def get_deployed_policies(audit):
@@ -43,12 +44,12 @@
if audit.is_not_found():
warning_txt = "got no deployed policies or policy-filters"
- PolicyMatcher._logger.warning(warning_txt)
+ _LOGGER.warning(warning_txt)
return {"warning": warning_txt}, None, None
if not audit.is_success() or (not deployed_policies and not deployed_policy_filters):
error_txt = "failed to retrieve policies from deployment-handler"
- PolicyMatcher._logger.error(error_txt)
+ _LOGGER.error(error_txt)
return {"error": error_txt}, None, None
return None, deployed_policies, deployed_policy_filters
@@ -62,7 +63,7 @@
if not (deployed_policies or deployed_policy_filters):
error_txt = "no deployed policies or policy-filters"
- PolicyMatcher._logger.warning(error_txt)
+ _LOGGER.warning(error_txt)
return {"error": error_txt}, None
coarse_regex_patterns = PolicyMatcher.calc_coarse_patterns(
@@ -72,7 +73,7 @@
error_txt = ("failed to construct the coarse_regex_patterns from " +
"deployed_policies: {} and deployed_policy_filters: {}"
.format(deployed_policies, deployed_policy_filters))
- PolicyMatcher._logger.error(audit.error(
+ _LOGGER.error(audit.error(
error_txt, error_code=AuditResponseCode.DATA_ERROR))
audit.set_http_status_code(AuditHttpCode.DATA_ERROR.value)
return {"error": error_txt}, None
@@ -84,7 +85,7 @@
if not audit.is_success():
error_txt = "failed to retrieve policies from policy-engine"
- PolicyMatcher._logger.warning(error_txt)
+ _LOGGER.warning(error_txt)
return {"error": error_txt}, None
latest_policies = pdp_response.get(LATEST_POLICIES, {})
@@ -123,7 +124,7 @@
coarse_regex.add_regex_pattern(policy_name_pattern)
coarse_regex_patterns = coarse_regex.get_coarse_regex_patterns()
- PolicyMatcher._logger.debug(
+ _LOGGER.debug(
audit.debug("coarse_regex_patterns({}) combined_regex_pattern({}) for patterns({})"
.format(coarse_regex_patterns,
coarse_regex.get_combined_regex_pattern(),
@@ -219,11 +220,11 @@
log_line = "policy {} to filter id {}: {}".format(json.dumps(policy),
policy_filter_id,
json.dumps(policy_filter))
- # PolicyMatcher._logger.debug(audit.debug("matching {}...".format(log_line)))
+ # _LOGGER.debug(audit.debug("matching {}...".format(log_line)))
if (filter_policy_name != policy_id and filter_policy_name != policy_name
and not re.match(filter_policy_name, policy_name)):
- PolicyMatcher._logger.debug(
+ _LOGGER.debug(
audit.debug("not match by policyName: {} != {}: {}"
.format(policy_name, filter_policy_name, log_line)))
return False
@@ -235,7 +236,7 @@
filter_onap_name = policy_filter.get("onapName")
policy_onap_name = matching_conditions.get("ONAPName")
if filter_onap_name and filter_onap_name != policy_onap_name:
- PolicyMatcher._logger.debug(
+ _LOGGER.debug(
audit.debug("not match by ONAPName: {} != {}: {}"
.format(policy_onap_name, filter_onap_name, log_line)))
return False
@@ -243,7 +244,7 @@
filter_config_name = policy_filter.get("configName")
policy_config_name = matching_conditions.get("ConfigName")
if filter_config_name and filter_config_name != policy_config_name:
- PolicyMatcher._logger.debug(
+ _LOGGER.debug(
audit.debug("not match by configName: {} != {}: {}"
.format(policy_config_name, filter_config_name, log_line)))
return False
@@ -253,12 +254,12 @@
for filter_key, filter_config_attribute in filter_config_attributes.items():
if (filter_key not in matching_conditions
or filter_config_attribute != matching_conditions.get(filter_key)):
- PolicyMatcher._logger.debug(
+ _LOGGER.debug(
audit.debug("not match by configAttributes: {} != {}: {}"
.format(json.dumps(matching_conditions),
json.dumps(filter_config_attributes),
log_line)))
return False
- PolicyMatcher._logger.debug(audit.debug("matched {}".format(log_line)))
+ _LOGGER.debug(audit.debug("matched {}".format(log_line)))
return True
diff --git a/policyhandler/policy_rest.py b/policyhandler/pdp_api_v0/policy_rest.py
similarity index 84%
rename from policyhandler/policy_rest.py
rename to policyhandler/pdp_api_v0/policy_rest.py
index 85dd914..c59625e 100644
--- a/policyhandler/policy_rest.py
+++ b/policyhandler/pdp_api_v0/policy_rest.py
@@ -1,5 +1,5 @@
# ================================================================================
-# Copyright (c) 2017-2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2017-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,13 +14,12 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""policy-client communicates with policy-engine thru REST API"""
import copy
import json
-import logging
+import os
import time
import urllib.parse
from multiprocessing.dummy import Pool as ThreadPool
@@ -28,18 +27,21 @@
import requests
-from .config import Config, Settings
-from .onap.audit import (REQUEST_X_ECOMP_REQUESTID, AuditHttpCode,
- AuditResponseCode, Metrics)
-from .policy_consts import (ERRORED_POLICIES, LATEST_POLICIES, POLICY_BODY,
- POLICY_CONFIG, POLICY_FILTER, POLICY_FILTERS,
- POLICY_ID, POLICY_NAME)
+from ..config import Config, Settings
+from ..onap.audit import (REQUEST_X_ECOMP_REQUESTID, AuditHttpCode,
+ AuditResponseCode, Metrics)
+from ..policy_consts import (ERRORED_POLICIES, LATEST_POLICIES, POLICY_BODY,
+ POLICY_FILTER, POLICY_FILTERS, POLICY_ID,
+ POLICY_NAMES)
+from ..utils import Utils
+from .pdp_consts import POLICY_CONFIG, POLICY_NAME, POLICY_VERSION
from .policy_utils import PolicyUtils
+_LOGGER = Utils.get_logger(__file__)
class PolicyRest(object):
"""using the http API to policy-engine"""
- _logger = logging.getLogger("policy_handler.policy_rest")
+ PDP_API_FOLDER = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
_lazy_inited = False
POLICY_GET_CONFIG = 'getConfig'
PDP_CONFIG_STATUS = "policyConfigStatus"
@@ -60,6 +62,7 @@
Config.POLICY_RETRY_COUNT, Config.POLICY_RETRY_SLEEP)
_requests_session = None
+ _url = None
_url_get_config = None
_headers = None
_target_entity = None
@@ -73,8 +76,7 @@
def _init():
"""init static config"""
PolicyRest._custom_kwargs = {}
-
- _, config = PolicyRest._settings.get_by_key(Config.FIELD_POLICY_ENGINE)
+ tls_ca_mode = None
if not PolicyRest._requests_session:
PolicyRest._requests_session = requests.Session()
@@ -88,28 +90,33 @@
'http://', requests.adapters.HTTPAdapter(pool_connections=pool_size,
pool_maxsize=pool_size))
- get_config_path = urllib.parse.urljoin(
- config.get("path_api", "pdp/api").strip("/") + "/", PolicyRest.POLICY_GET_CONFIG)
- PolicyRest._url_get_config = urllib.parse.urljoin(config.get("url", ""), get_config_path)
- PolicyRest._headers = config.get("headers", {})
- PolicyRest._target_entity = config.get("target_entity", Config.FIELD_POLICY_ENGINE)
- _, PolicyRest._thread_pool_size = PolicyRest._settings.get_by_key(
- Config.THREAD_POOL_SIZE, 4)
- if PolicyRest._thread_pool_size < 2:
- PolicyRest._thread_pool_size = 2
+ _, config = PolicyRest._settings.get_by_key(Config.FIELD_POLICY_ENGINE)
+ if config:
+ PolicyRest._url = config.get("url")
+ if PolicyRest._url:
+ path_get_config = urllib.parse.urljoin(
+ config.get("path_api", "pdp/api").strip("/")
+ + "/", PolicyRest.POLICY_GET_CONFIG)
+ PolicyRest._url_get_config = urllib.parse.urljoin(PolicyRest._url, path_get_config)
+ PolicyRest._headers = config.get("headers", {})
+ PolicyRest._target_entity = config.get("target_entity", Config.FIELD_POLICY_ENGINE)
+ _, PolicyRest._thread_pool_size = PolicyRest._settings.get_by_key(
+ Config.THREAD_POOL_SIZE, 4)
+ if PolicyRest._thread_pool_size < 2:
+ PolicyRest._thread_pool_size = 2
- _, PolicyRest._policy_retry_count = PolicyRest._settings.get_by_key(
- Config.POLICY_RETRY_COUNT, 1)
- _, PolicyRest._policy_retry_sleep = PolicyRest._settings.get_by_key(
- Config.POLICY_RETRY_SLEEP, 0)
+ _, PolicyRest._policy_retry_count = PolicyRest._settings.get_by_key(
+ Config.POLICY_RETRY_COUNT, 1)
+ _, PolicyRest._policy_retry_sleep = PolicyRest._settings.get_by_key(
+ Config.POLICY_RETRY_SLEEP, 0)
- tls_ca_mode = config.get(Config.TLS_CA_MODE)
- PolicyRest._custom_kwargs = Config.get_requests_kwargs(tls_ca_mode)
- PolicyRest._timeout_in_secs = config.get(Config.TIMEOUT_IN_SECS)
- if not PolicyRest._timeout_in_secs or PolicyRest._timeout_in_secs < 1:
- PolicyRest._timeout_in_secs = PolicyRest.DEFAULT_TIMEOUT_IN_SECS
+ tls_ca_mode = config.get(Config.TLS_CA_MODE)
+ PolicyRest._custom_kwargs = Config.get_requests_kwargs(tls_ca_mode)
+ PolicyRest._timeout_in_secs = config.get(Config.TIMEOUT_IN_SECS)
+ if not PolicyRest._timeout_in_secs or PolicyRest._timeout_in_secs < 1:
+ PolicyRest._timeout_in_secs = PolicyRest.DEFAULT_TIMEOUT_IN_SECS
- PolicyRest._logger.info(
+ _LOGGER.info(
"PDP(%s) url(%s) headers(%s) tls_ca_mode(%s) timeout_in_secs(%s) custom_kwargs(%s): %s",
PolicyRest._target_entity, PolicyRest._url_get_config,
Metrics.json_dumps(PolicyRest._headers), tls_ca_mode,
@@ -148,6 +155,12 @@
@staticmethod
def _pdp_get_config(audit, json_body):
"""Communication with the policy-engine"""
+ if not PolicyRest._url:
+ _LOGGER.error(
+ audit.error("no url for PDP", error_code=AuditResponseCode.AVAILABILITY_ERROR))
+ audit.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
+ return None
+
with PolicyRest._lock:
session = PolicyRest._requests_session
target_entity = PolicyRest._target_entity
@@ -166,7 +179,7 @@
timeout_in_secs)
log_line = log_action + " " + log_data
- PolicyRest._logger.info(metrics.metrics_start(log_line))
+ _LOGGER.info(metrics.metrics_start(log_line))
res = None
try:
@@ -178,7 +191,7 @@
else AuditHttpCode.SERVER_INTERNAL_ERROR.value)
error_msg = ("failed {}: {} to {}".format(type(ex).__name__, str(ex), log_line))
- PolicyRest._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
metrics.set_http_status_code(error_code)
audit.set_http_status_code(error_code)
metrics.metrics(error_msg)
@@ -195,7 +208,7 @@
metrics.set_http_status_code(res.status_code)
metrics.metrics(log_line)
- PolicyRest._logger.info(log_line)
+ _LOGGER.info(log_line)
return res.status_code, res_data
@staticmethod
@@ -213,7 +226,7 @@
error_code = AuditHttpCode.SERVICE_UNAVAILABLE_ERROR.value
error_msg = "{} unexpected {}".format(error_code, log_line)
- PolicyRest._logger.error(error_msg)
+ _LOGGER.error(error_msg)
metrics.set_http_status_code(error_code)
audit.set_http_status_code(error_code)
metrics.metrics(error_msg)
@@ -236,7 +249,7 @@
status_code = AuditHttpCode.DATA_NOT_FOUND_OK.value
info_msg = "{} not found {}".format(status_code, log_line)
- PolicyRest._logger.info(info_msg)
+ _LOGGER.info(info_msg)
metrics.set_http_status_code(status_code)
metrics.metrics(info_msg)
return status_code, None
@@ -273,7 +286,7 @@
.format(audit.request_id, type(ex).__name__, str(ex),
"get_latest_policy", str_metrics))
- PolicyRest._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
audit.fatal(error_msg, error_code=AuditResponseCode.BUSINESS_PROCESS_ERROR)
audit.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
return None
@@ -290,8 +303,7 @@
expect_policy_removed = (ignore_policy_names and not expected_versions)
for retry in range(1, PolicyRest._policy_retry_count + 1):
- PolicyRest._logger.debug("try(%s) retry_get_config(%s): %s",
- retry, retry_get_config, str_metrics)
+ _LOGGER.debug("try(%s) retry_get_config(%s): %s", retry, retry_get_config, str_metrics)
done, latest_policy, status_code = PolicyRest._get_latest_policy_once(
audit, policy_id, expected_versions, ignore_policy_names,
@@ -301,13 +313,13 @@
break
if retry == PolicyRest._policy_retry_count:
- PolicyRest._logger.error(
+ _LOGGER.error(
audit.error("gave up retrying after #{} for policy_id({}) from PDP {}"
.format(retry, policy_id, PolicyRest._url_get_config),
error_code=AuditResponseCode.DATA_ERROR))
break
- PolicyRest._logger.warning(audit.warn(
+ _LOGGER.warning(audit.warn(
"will retry({}) for policy_id({}) in {} secs from PDP {}".format(
retry, policy_id, PolicyRest._policy_retry_sleep, PolicyRest._url_get_config),
error_code=AuditResponseCode.DATA_ERROR))
@@ -321,7 +333,7 @@
audit.set_http_status_code(status_code)
if not PolicyRest._validate_policy(latest_policy):
audit.set_http_status_code(AuditHttpCode.DATA_NOT_FOUND_OK.value)
- PolicyRest._logger.error(audit.error(
+ _LOGGER.error(audit.error(
"received invalid policy from PDP: {}".format(json.dumps(latest_policy)),
error_code=AuditResponseCode.DATA_ERROR))
@@ -335,15 +347,15 @@
status_code, policy_bodies = PolicyRest._pdp_get_config(audit, {POLICY_NAME:policy_id})
- PolicyRest._logger.debug("%s %s policy_bodies: %s",
- status_code, policy_id, json.dumps(policy_bodies or []))
+ _LOGGER.debug("%s %s policy_bodies: %s",
+ status_code, policy_id, json.dumps(policy_bodies or []))
latest_policy = PolicyUtils.select_latest_policy(
policy_bodies, expected_versions, ignore_policy_names
)
if not latest_policy and not expect_policy_removed:
- PolicyRest._logger.error(
+ _LOGGER.error(
audit.error("received unexpected policy data from PDP for policy_id={}: {}"
.format(policy_id, json.dumps(policy_bodies or [])),
error_code=AuditResponseCode.DATA_ERROR))
@@ -355,9 +367,16 @@
return done, latest_policy, status_code
@staticmethod
- def get_latest_updated_policies(aud_policy_updates):
+ def get_latest_updated_policies(audit, updated_policies, removed_policies):
"""safely try retrieving the latest policies for the list of policy_names"""
- audit, policies_updated, policies_removed = aud_policy_updates
+ if not updated_policies and not removed_policies:
+ return None, None
+
+ policies_updated = [(policy_id, policy.get(POLICY_BODY, {}).get(POLICY_VERSION))
+ for policy_id, policy in updated_policies.items()]
+ policies_removed = [(policy_id, policy.get(POLICY_NAMES, {}))
+ for policy_id, policy in removed_policies.items()]
+
if not policies_updated and not policies_removed:
return None, None
@@ -374,7 +393,7 @@
.format(audit.request_id, type(ex).__name__, str(ex),
"get_latest_updated_policies", str_metrics))
- PolicyRest._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
audit.fatal(error_msg, error_code=AuditResponseCode.BUSINESS_PROCESS_ERROR)
audit.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
return None, None
@@ -389,7 +408,7 @@
targetServiceName=PolicyRest._url_get_config)
metrics_total.metrics_start("get_latest_updated_policies {0}".format(str_metrics))
- PolicyRest._logger.debug(str_metrics)
+ _LOGGER.debug(str_metrics)
policies_to_find = {}
for (policy_id, policy_version) in policies_updated:
@@ -424,8 +443,8 @@
policies = None
apns_length = len(apns)
- PolicyRest._logger.debug("apns_length(%s) policies_to_find %s", apns_length,
- json.dumps(policies_to_find))
+ _LOGGER.debug("apns_length(%s) policies_to_find %s", apns_length,
+ json.dumps(policies_to_find))
if apns_length == 1:
policies = [PolicyRest.get_latest_policy(apns[0])]
@@ -454,7 +473,7 @@
if policy_id not in updated_policies
and policy_id not in removed_policies)
- PolicyRest._logger.debug(
+ _LOGGER.debug(
"result(%s) updated_policies %s, removed_policies %s, errored_policies %s",
apns_length, json.dumps(updated_policies), json.dumps(removed_policies),
json.dumps(errored_policies))
@@ -474,19 +493,19 @@
audit, policy_filter = aud_policy_filter
try:
str_policy_filter = json.dumps(policy_filter)
- PolicyRest._logger.debug("%s", str_policy_filter)
+ _LOGGER.debug("%s", str_policy_filter)
status_code, policy_bodies = PolicyRest._pdp_get_config(audit, policy_filter)
audit.set_http_status_code(status_code)
- PolicyRest._logger.debug("%s policy_bodies: %s %s", status_code,
- str_policy_filter, json.dumps(policy_bodies or []))
+ _LOGGER.debug("%s policy_bodies: %s %s", status_code,
+ str_policy_filter, json.dumps(policy_bodies or []))
latest_policies = PolicyUtils.select_latest_policies(policy_bodies)
if not latest_policies:
audit.set_http_status_code(AuditHttpCode.DATA_NOT_FOUND_OK.value)
- PolicyRest._logger.warning(audit.warn(
+ _LOGGER.warning(audit.warn(
"received no policies from PDP for policy_filter {}: {}"
.format(str_policy_filter, json.dumps(policy_bodies or [])),
error_code=AuditResponseCode.DATA_ERROR))
@@ -506,7 +525,7 @@
.format(audit.request_id, type(ex).__name__, str(ex),
"_get_latest_policies", json.dumps(policy_filter)))
- PolicyRest._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
audit.fatal(error_msg, error_code=AuditResponseCode.BUSINESS_PROCESS_ERROR)
audit.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
return None, None
@@ -545,7 +564,7 @@
else:
return result
- PolicyRest._logger.debug("%s", str_policy_filters)
+ _LOGGER.debug("%s", str_policy_filters)
metrics_total = Metrics(aud_parent=audit, targetEntity=target_entity,
targetServiceName=PolicyRest._url_get_config)
@@ -571,8 +590,8 @@
result[ERRORED_POLICIES] = dict(
pair for (_, eps) in latest_policies if eps for pair in eps.items())
- PolicyRest._logger.debug("got policies for policy_filters: %s. result: %s",
- str_policy_filters, json.dumps(result))
+ _LOGGER.debug("got policies for policy_filters: %s. result: %s",
+ str_policy_filters, json.dumps(result))
return result
except Exception as ex:
@@ -580,7 +599,7 @@
.format(audit.request_id, type(ex).__name__, str(ex),
"get_latest_policies", str_metrics))
- PolicyRest._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
audit.fatal(error_msg, error_code=AuditResponseCode.BUSINESS_PROCESS_ERROR)
audit.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
return None
diff --git a/policyhandler/pdp_api_v0/policy_updates.py b/policyhandler/pdp_api_v0/policy_updates.py
new file mode 100644
index 0000000..eafdca2
--- /dev/null
+++ b/policyhandler/pdp_api_v0/policy_updates.py
@@ -0,0 +1,107 @@
+# ================================================================================
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""policy-updates accumulates the policy-update notifications from PDP"""
+
+import json
+import os
+
+from ..onap.audit import Audit
+from ..policy_consts import POLICY_ID, POLICY_NAMES
+from ..utils import Utils
+from .pdp_consts import POLICY_NAME
+from .policy_utils import PolicyUtils
+
+
+_LOGGER = Utils.get_logger(__file__)
+
+class PolicyUpdates(object):
+ """Keep and consolidate the policy-updates (audit, policies_updated, policies_removed)"""
+ PDP_API_FOLDER = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
+
+ def __init__(self):
+ """init and reset"""
+ self._audit = None
+ self._policies_updated = {}
+ self._policies_removed = {}
+
+ def reset(self):
+ """resets the state"""
+ self.__init__()
+
+ def pop_policy_updates(self):
+ """
+ Returns the consolidated (audit, policies_updated, policies_removed)
+ and resets the state
+ """
+ if not self._audit:
+ return None, None, None
+
+ audit = self._audit
+ policies_updated = self._policies_updated
+ policies_removed = self._policies_removed
+
+ self.reset()
+
+ return audit, policies_updated, policies_removed
+
+
+ def push_policy_updates(self, policies_updated, policies_removed):
+ """consolidate the new policies_updated, policies_removed to existing ones"""
+ for policy_body in policies_updated:
+ policy_name = policy_body.get(POLICY_NAME)
+ policy = PolicyUtils.convert_to_policy(policy_body)
+ if not policy:
+ continue
+ policy_id = policy.get(POLICY_ID)
+
+ self._policies_updated[policy_id] = policy
+
+ rm_policy_names = self._policies_removed.get(policy_id, {}).get(POLICY_NAMES)
+ if rm_policy_names and policy_name in rm_policy_names:
+ del rm_policy_names[policy_name]
+
+ for policy_body in policies_removed:
+ policy_name = policy_body.get(POLICY_NAME)
+ policy = PolicyUtils.convert_to_policy(policy_body)
+ if not policy:
+ continue
+ policy_id = policy.get(POLICY_ID)
+
+ if policy_id in self._policies_removed:
+ policy = self._policies_removed[policy_id]
+
+ if POLICY_NAMES not in policy:
+ policy[POLICY_NAMES] = {}
+ policy[POLICY_NAMES][policy_name] = True
+ self._policies_removed[policy_id] = policy
+
+ req_message = ("policy-update notification - updated[{0}], removed[{1}]"
+ .format(len(self._policies_updated),
+ len(self._policies_removed)))
+
+ if not self._audit:
+ self._audit = Audit(job_name="policy_update",
+ req_message=req_message,
+ retry_get_config=True)
+ else:
+ self._audit.req_message = req_message
+
+ _LOGGER.info(
+ "pending(%s) for %s policies_updated %s policies_removed %s",
+ self._audit.request_id, req_message,
+ json.dumps(self._policies_updated), json.dumps(self._policies_removed))
diff --git a/policyhandler/pdp_api_v0/policy_utils.py b/policyhandler/pdp_api_v0/policy_utils.py
new file mode 100644
index 0000000..d337665
--- /dev/null
+++ b/policyhandler/pdp_api_v0/policy_utils.py
@@ -0,0 +1,120 @@
+# ================================================================================
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""utils for policy usage and conversions"""
+
+import re
+
+from ..policy_consts import POLICY_BODY, POLICY_ID
+from ..utils import Utils
+from .pdp_consts import POLICY_CONFIG, POLICY_NAME, POLICY_VERSION
+
+
+class PolicyUtils(object):
+ """policy-client utils"""
+ _policy_name_ext = re.compile('[.][0-9]+[.][a-zA-Z]+$')
+
+ @staticmethod
+ def extract_policy_id(policy_name):
+ """ policy_name = policy_id + "." + <version> + "." + <extension>
+ For instance,
+ policy_name = DCAE_alex.Config_alex_policy_number_1.3.xml
+ policy_id = DCAE_alex.Config_alex_policy_number_1
+ policy_scope = DCAE_alex
+ policy_class = Config
+ policy_version = 3
+ type = extension = xml
+ delimiter = "."
+ policy_class_delimiter = "_"
+ policy_name in PAP = DCAE_alex.alex_policy_number_1
+ """
+ if not policy_name:
+ return
+ return PolicyUtils._policy_name_ext.sub('', policy_name)
+
+ @staticmethod
+ def parse_policy_config(policy):
+ """try parsing the config in policy."""
+ if not policy:
+ return policy
+ config = policy.get(POLICY_BODY, {}).get(POLICY_CONFIG)
+ if config:
+ policy[POLICY_BODY][POLICY_CONFIG] = Utils.safe_json_parse(config)
+ return policy
+
+ @staticmethod
+ def convert_to_policy(policy_body):
+ """wrap policy_body received from policy-engine with policy_id."""
+ if not policy_body:
+ return None
+ policy_name = policy_body.get(POLICY_NAME)
+ policy_version = policy_body.get(POLICY_VERSION)
+ if not policy_name or not policy_version:
+ return None
+ policy_id = PolicyUtils.extract_policy_id(policy_name)
+ if not policy_id:
+ return None
+ return {POLICY_ID:policy_id, POLICY_BODY:policy_body}
+
+ @staticmethod
+ def select_latest_policy(policy_bodies, expected_versions=None, ignore_policy_names=None):
+ """For some reason, the policy-engine returns all version of the policy_bodies.
+ DCAE-Controller is only interested in the latest version
+ """
+ if not policy_bodies:
+ return
+ latest_policy_body = {}
+ for policy_body in policy_bodies:
+ policy_name = policy_body.get(POLICY_NAME)
+ policy_version = policy_body.get(POLICY_VERSION)
+ if not policy_name or not policy_version or not policy_version.isdigit():
+ continue
+ if expected_versions and policy_version not in expected_versions:
+ continue
+ if ignore_policy_names and policy_name in ignore_policy_names:
+ continue
+
+ if (not latest_policy_body
+ or int(latest_policy_body[POLICY_VERSION]) < int(policy_version)):
+ latest_policy_body = policy_body
+
+ return PolicyUtils.parse_policy_config(PolicyUtils.convert_to_policy(latest_policy_body))
+
+ @staticmethod
+ def select_latest_policies(policy_bodies):
+ """For some reason, the policy-engine returns all version of the policy_bodies.
+ DCAE-Controller is only interested in the latest versions
+ """
+ if not policy_bodies:
+ return {}
+ policies = {}
+ for policy_body in policy_bodies:
+ policy = PolicyUtils.convert_to_policy(policy_body)
+ if not policy:
+ continue
+ policy_id = policy.get(POLICY_ID)
+ policy_version = policy.get(POLICY_BODY, {}).get(POLICY_VERSION)
+ if not policy_id or not policy_version or not policy_version.isdigit():
+ continue
+ if (policy_id not in policies
+ or int(policy_version) > int(policies[policy_id][POLICY_BODY][POLICY_VERSION])):
+ policies[policy_id] = policy
+
+ for policy_id in policies:
+ policies[policy_id] = PolicyUtils.parse_policy_config(policies[policy_id])
+
+ return policies
diff --git a/policyhandler/pdp_client.py b/policyhandler/pdp_client.py
new file mode 100644
index 0000000..353b604
--- /dev/null
+++ b/policyhandler/pdp_client.py
@@ -0,0 +1,29 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""policy-client communicates with policy-engine thru REST API"""
+
+from .config import Config
+from .utils import Utils
+
+if Config.is_pdp_api_default():
+ from .pdp_api import *
+else:
+ from .pdp_api_v0 import *
+
+_LOGGER = Utils.get_logger(__file__)
+_LOGGER.info(get_pdp_api_info())
diff --git a/policyhandler/policy_consts.py b/policyhandler/policy_consts.py
index 8f3ec76..2eafc53 100644
--- a/policyhandler/policy_consts.py
+++ b/policyhandler/policy_consts.py
@@ -14,15 +14,11 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""contants of policy-handler"""
POLICY_ID = 'policy_id'
-POLICY_VERSION = "policyVersion"
-POLICY_NAME = "policyName"
POLICY_BODY = 'policy_body'
-POLICY_CONFIG = 'config'
CATCH_UP = "catch_up"
AUTO_CATCH_UP = "auto catch_up"
@@ -34,7 +30,6 @@
POLICY_FILTERS = "policy_filters"
POLICIES = "policies"
POLICY_VERSIONS = "policy_versions"
-MATCHING_CONDITIONS = "matchingConditions"
POLICY_NAMES = "policy_names"
POLICY_FILTER_MATCHES = "policy_filter_matches"
TARGET_ENTITY = "target_entity"
diff --git a/policyhandler/policy_receiver.py b/policyhandler/policy_receiver.py
index 7cf1869..d949c4b 100644
--- a/policyhandler/policy_receiver.py
+++ b/policyhandler/policy_receiver.py
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""
policy-receiver communicates with policy-engine
@@ -25,323 +24,37 @@
passes the notifications to policy-updater
"""
-import copy
-import json
-import logging
-import ssl
-import time
-import urllib.parse
-from datetime import datetime
-from threading import Lock, Thread
-
-import websocket
-
-from .config import Config, Settings
-from .onap.audit import Audit
-from .policy_consts import MATCHING_CONDITIONS, POLICY_NAME, POLICY_VERSION
-from .policy_updater import PolicyUpdater
-from .policy_utils import Utils
from .service_activator import ServiceActivator
-LOADED_POLICIES = 'loadedPolicies'
-REMOVED_POLICIES = 'removedPolicies'
-POLICY_VER = 'versionNo'
-POLICY_MATCHES = 'matches'
-
-class _PolicyReceiver(Thread):
- """web-socket to PolicyEngine"""
- _logger = logging.getLogger("policy_handler.policy_receiver")
- WS_STARTED = "started"
- WS_START_COUNT = "start_count"
- WS_CLOSE_COUNT = "close_count"
- WS_ERROR_COUNT = "error_count"
- WS_PONG_COUNT = "pong_count"
- WS_MESSAGE_COUNT = "message_count"
- WS_MESSAGE_TIMESTAMP = "message_timestamp"
- WS_STATUS = "status"
- WS_PING_INTERVAL_DEFAULT = 30
- WEB_SOCKET_HEALTH = "web_socket_health"
-
- def __init__(self, audit, policy_updater):
- """web-socket inside the thread to receive policy notifications from PolicyEngine"""
- Thread.__init__(self, name="policy_receiver", daemon=True)
-
- self._policy_updater = policy_updater
- self._lock = Lock()
- self._keep_running = True
- self._settings = Settings(Config.FIELD_POLICY_ENGINE)
-
- self._sleep_before_restarting = 5
- self._web_socket_url = None
- self._web_socket_sslopt = None
- self._tls_wss_ca_mode = None
- self._web_socket = None
- self._ws_ping_interval_in_secs = _PolicyReceiver.WS_PING_INTERVAL_DEFAULT
- self._web_socket_health = {
- _PolicyReceiver.WS_START_COUNT: 0,
- _PolicyReceiver.WS_CLOSE_COUNT: 0,
- _PolicyReceiver.WS_ERROR_COUNT: 0,
- _PolicyReceiver.WS_PONG_COUNT: 0,
- _PolicyReceiver.WS_MESSAGE_COUNT: 0,
- _PolicyReceiver.WS_STATUS: "created"
- }
-
- Audit.register_item_health(_PolicyReceiver.WEB_SOCKET_HEALTH, self._get_health)
- self.reconfigure(audit)
-
- def reconfigure(self, audit):
- """configure and reconfigure the web-socket"""
- with self._lock:
- _PolicyReceiver._logger.info(audit.info("web_socket_health {}".format(
- json.dumps(self._get_health(), sort_keys=True))))
- self._sleep_before_restarting = 5
- self._settings.set_config(Config.discovered_config)
- changed, config = self._settings.get_by_key(Config.FIELD_POLICY_ENGINE)
-
- if not changed:
- self._settings.commit_change()
- return False
-
- prev_web_socket_url = self._web_socket_url
- prev_web_socket_sslopt = self._web_socket_sslopt
- prev_ws_ping_interval_in_secs = self._ws_ping_interval_in_secs
-
- self._web_socket_sslopt = None
-
- resturl = urllib.parse.urljoin(config.get("url", "").lower().rstrip("/") + "/",
- config.get("path_notifications", "/pdp/notifications"))
-
- self._tls_wss_ca_mode = config.get(Config.TLS_WSS_CA_MODE)
-
- self._ws_ping_interval_in_secs = config.get(Config.WS_PING_INTERVAL_IN_SECS)
- if not self._ws_ping_interval_in_secs or self._ws_ping_interval_in_secs < 60:
- self._ws_ping_interval_in_secs = _PolicyReceiver.WS_PING_INTERVAL_DEFAULT
-
- if resturl.startswith("https:"):
- self._web_socket_url = resturl.replace("https:", "wss:")
-
- verify = Config.get_tls_verify(self._tls_wss_ca_mode)
- if verify is False:
- self._web_socket_sslopt = {'cert_reqs': ssl.CERT_NONE}
- elif verify is True:
- pass
- else:
- self._web_socket_sslopt = {'ca_certs': verify}
-
- else:
- self._web_socket_url = resturl.replace("http:", "ws:")
-
- log_changed = (
- "changed web_socket_url(%s) or tls_wss_ca_mode(%s)"
- " or ws_ping_interval_in_secs(%s): %s" %
- (self._web_socket_url, self._tls_wss_ca_mode, self._ws_ping_interval_in_secs,
- self._settings))
- if (self._web_socket_url == prev_web_socket_url
- and Utils.are_the_same(prev_web_socket_sslopt, self._web_socket_sslopt)
- and prev_ws_ping_interval_in_secs == self._ws_ping_interval_in_secs):
- _PolicyReceiver._logger.info(audit.info("not {}".format(log_changed)))
- self._settings.commit_change()
- return False
-
- _PolicyReceiver._logger.info(audit.info(log_changed))
- self._settings.commit_change()
-
- self._stop_notifications()
- return True
-
- def run(self):
- """listen on web-socket and pass the policy notifications to policy-updater"""
- _PolicyReceiver._logger.info("starting policy_receiver...")
- websocket.enableTrace(True)
- restarting = False
- while True:
- if not self._get_keep_running():
- break
-
- self._stop_notifications()
-
- if restarting:
- with self._lock:
- sleep_before_restarting = self._sleep_before_restarting
- _PolicyReceiver._logger.info(
- "going to sleep for %s secs before restarting policy-notifications",
- sleep_before_restarting)
-
- time.sleep(sleep_before_restarting)
- if not self._get_keep_running():
- break
-
- with self._lock:
- web_socket_url = self._web_socket_url
- sslopt = copy.deepcopy(self._web_socket_sslopt)
- tls_wss_ca_mode = self._tls_wss_ca_mode
- ws_ping_interval_in_secs = self._ws_ping_interval_in_secs
-
- _PolicyReceiver._logger.info(
- "connecting to policy-notifications at %s with sslopt(%s) tls_wss_ca_mode(%s)"
- " ws_ping_interval_in_secs(%s)",
- web_socket_url, json.dumps(sslopt), tls_wss_ca_mode, ws_ping_interval_in_secs)
-
- self._web_socket = websocket.WebSocketApp(
- web_socket_url,
- on_open=self._on_ws_open,
- on_message=self._on_pdp_message,
- on_close=self._on_ws_close,
- on_error=self._on_ws_error,
- on_pong=self._on_ws_pong
- )
-
- _PolicyReceiver._logger.info("waiting for policy-notifications...")
- self._web_socket.run_forever(sslopt=sslopt, ping_interval=ws_ping_interval_in_secs)
- restarting = True
-
- Audit.register_item_health(_PolicyReceiver.WEB_SOCKET_HEALTH)
- _PolicyReceiver._logger.info("exit policy-receiver")
-
- def _get_keep_running(self):
- """thread-safe check whether to continue running"""
- with self._lock:
- keep_running = self._keep_running
- return keep_running
-
- def _stop_notifications(self):
- """close the web-socket == stops the notification service if running."""
- with self._lock:
- if self._web_socket and self._web_socket.sock and self._web_socket.sock.connected:
- self._web_socket.close()
- _PolicyReceiver._logger.info("stopped receiving notifications from PDP")
-
- def _on_pdp_message(self, *args):
- """received the notification from PDP"""
- self._web_socket_health[_PolicyReceiver.WS_MESSAGE_COUNT] += 1
- self._web_socket_health[_PolicyReceiver.WS_MESSAGE_TIMESTAMP] = str(datetime.utcnow())
- try:
- message = args and args[-1]
- _PolicyReceiver._logger.info("Received notification message: %s", message)
- _PolicyReceiver._logger.info("web_socket_health %s",
- json.dumps(self._get_health(), sort_keys=True))
- if not message:
- return
- message = json.loads(message)
-
- if not message or not isinstance(message, dict):
- _PolicyReceiver._logger.warning("unexpected message from PDP: %s",
- json.dumps(message))
- return
-
- policies_updated = [
- {POLICY_NAME: policy.get(POLICY_NAME),
- POLICY_VERSION: policy.get(POLICY_VER),
- MATCHING_CONDITIONS: policy.get(POLICY_MATCHES, {})}
- for policy in message.get(LOADED_POLICIES, [])
- ]
-
- policies_removed = [
- {POLICY_NAME: removed_policy.get(POLICY_NAME),
- POLICY_VERSION: removed_policy.get(POLICY_VER)}
- for removed_policy in message.get(REMOVED_POLICIES, [])
- ]
-
- if not policies_updated and not policies_removed:
- _PolicyReceiver._logger.info("no policy updated or removed")
- return
-
- self._policy_updater.policy_update(policies_updated, policies_removed)
- except Exception as ex:
- error_msg = "crash {} {} at {}: {}".format(type(ex).__name__, str(ex),
- "on_pdp_message", json.dumps(message))
-
- _PolicyReceiver._logger.exception(error_msg)
-
- def _on_ws_error(self, error):
- """report an error"""
- _PolicyReceiver._logger.exception("policy-notification error %s", str(error))
- self._sleep_before_restarting = 60 if isinstance(error, ssl.SSLError) else 5
-
- self._web_socket_health[_PolicyReceiver.WS_STATUS] = "error"
- self._web_socket_health[_PolicyReceiver.WS_ERROR_COUNT] += 1
- self._web_socket_health["last_error"] = {
- "error": str(error), "timestamp": str(datetime.utcnow())
- }
- _PolicyReceiver._logger.info("web_socket_health %s",
- json.dumps(self._get_health(), sort_keys=True))
-
- def _on_ws_close(self, code, reason):
- """restart web-socket on close"""
- self._web_socket_health["last_closed"] = str(datetime.utcnow())
- self._web_socket_health[_PolicyReceiver.WS_STATUS] = "closed"
- self._web_socket_health[_PolicyReceiver.WS_CLOSE_COUNT] += 1
- _PolicyReceiver._logger.info(
- "lost connection(%s, %s) to PDP web_socket_health %s",
- code, reason, json.dumps(self._get_health(), sort_keys=True))
-
- def _on_ws_open(self):
- """started web-socket"""
- self._web_socket_health[_PolicyReceiver.WS_STATUS] = _PolicyReceiver.WS_STARTED
- self._web_socket_health[_PolicyReceiver.WS_START_COUNT] += 1
- self._web_socket_health[_PolicyReceiver.WS_STARTED] = datetime.utcnow()
- _PolicyReceiver._logger.info("opened connection to PDP web_socket_health %s",
- json.dumps(self._get_health(), sort_keys=True))
-
- def _on_ws_pong(self, pong):
- """pong = response to pinging the server of the web-socket"""
- self._web_socket_health[_PolicyReceiver.WS_PONG_COUNT] += 1
- _PolicyReceiver._logger.info(
- "pong(%s) from connection to PDP web_socket_health %s",
- pong, json.dumps(self._get_health(), sort_keys=True))
-
- def _get_health(self):
- """returns the healthcheck of the web-socket as json"""
- web_socket_health = copy.deepcopy(self._web_socket_health)
- web_socket_health[Config.WS_PING_INTERVAL_IN_SECS] = self._ws_ping_interval_in_secs
- started = web_socket_health.get(_PolicyReceiver.WS_STARTED)
- if started:
- web_socket_health[_PolicyReceiver.WS_STARTED] = str(started)
- web_socket_health["uptime"] = str(datetime.utcnow() - started)
- return web_socket_health
-
-
- def shutdown(self, audit):
- """Shutdown the policy-receiver"""
- _PolicyReceiver._logger.info(audit.info("shutdown policy-receiver"))
- with self._lock:
- self._keep_running = False
-
- self._stop_notifications()
-
- if self.is_alive():
- self.join()
-
-
class PolicyReceiver(object):
"""
policy-receiver - static singleton wrapper around two threads
policy_updater - master thread for all scheduled actions
- policy_receiver - listens to policy-engine through web-socket
+ policy_listener - listens to policy-engine through web-socket
"""
_policy_updater = None
- _policy_receiver = None
+ _policy_listener = None
@staticmethod
def is_running():
"""check whether the policy-receiver runs"""
- return (PolicyReceiver._policy_receiver
- and PolicyReceiver._policy_receiver.is_alive()
+ return (PolicyReceiver._policy_listener
+ and PolicyReceiver._policy_listener.is_alive()
and PolicyReceiver._policy_updater
and PolicyReceiver._policy_updater.is_alive())
@staticmethod
- def _close_receiver(audit):
+ def _close_listener(audit):
"""stop the notification-handler"""
- if PolicyReceiver._policy_receiver:
- policy_receiver = PolicyReceiver._policy_receiver
- PolicyReceiver._policy_receiver = None
+ if PolicyReceiver._policy_listener:
+ policy_receiver = PolicyReceiver._policy_listener
+ PolicyReceiver._policy_listener = None
policy_receiver.shutdown(audit)
@staticmethod
def shutdown(audit):
"""shutdown the notification-handler and policy-updater"""
- PolicyReceiver._close_receiver(audit)
+ PolicyReceiver._close_listener(audit)
PolicyReceiver._policy_updater.shutdown(audit)
@staticmethod
@@ -359,23 +72,26 @@
"""act on reconfiguration event"""
active = ServiceActivator.is_active_mode_of_operation(audit)
- if not PolicyReceiver._policy_receiver:
+ if not PolicyReceiver._policy_listener:
if active:
- PolicyReceiver._policy_receiver = _PolicyReceiver(audit,
- PolicyReceiver._policy_updater)
- PolicyReceiver._policy_receiver.start()
+ from . import pdp_client
+ PolicyReceiver._policy_listener = pdp_client.PolicyListener(
+ audit, PolicyReceiver._policy_updater
+ )
+ PolicyReceiver._policy_listener.start()
return
if not active:
- PolicyReceiver._close_receiver(audit)
+ PolicyReceiver._close_listener(audit)
return
- PolicyReceiver._policy_receiver.reconfigure(audit)
+ PolicyReceiver._policy_listener.reconfigure(audit)
@staticmethod
def run(audit):
"""run policy_updater and policy_receiver"""
+ from .policy_updater import PolicyUpdater
PolicyReceiver._policy_updater = PolicyUpdater(PolicyReceiver._on_reconfigure)
PolicyReceiver._on_reconfigure(audit)
diff --git a/policyhandler/policy_updater.py b/policyhandler/policy_updater.py
index af1ea4b..3fcde40 100644
--- a/policyhandler/policy_updater.py
+++ b/policyhandler/policy_updater.py
@@ -14,109 +14,26 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""policy-updater thread"""
import json
-import logging
from threading import Event, Lock, Thread
+from . import pdp_client
from .config import Config, Settings
from .deploy_handler import DeployHandler, PolicyUpdateMessage
from .onap.audit import Audit, AuditHttpCode, AuditResponseCode
-from .policy_consts import (AUTO_CATCH_UP, AUTO_RECONFIGURE, CATCH_UP,
- POLICY_BODY, POLICY_ID, POLICY_NAME, POLICY_NAMES,
- POLICY_VERSION)
-from .policy_matcher import PolicyMatcher
-from .policy_rest import PolicyRest
-from .policy_utils import PolicyUtils
+from .policy_consts import AUTO_CATCH_UP, AUTO_RECONFIGURE, CATCH_UP
from .service_activator import ServiceActivator
from .step_timer import StepTimer
+from .utils import Utils
-class _PolicyUpdate(object):
- """Keep and consolidate the policy-updates (audit, policies_updated, policies_removed)"""
- _logger = logging.getLogger("policy_handler.policy_update")
-
- def __init__(self):
- """init and reset"""
- self._audit = None
- self._policies_updated = {}
- self._policies_removed = {}
-
- def reset(self):
- """resets the state"""
- self.__init__()
-
- def pop_policy_update(self):
- """
- Returns the consolidated (audit, policies_updated, policies_removed)
- and resets the state
- """
- if not self._audit:
- return None, None, None
-
- audit = self._audit
- policies_updated = self._policies_updated
- policies_removed = self._policies_removed
-
- self.reset()
-
- return audit, policies_updated, policies_removed
-
-
- def push_policy_update(self, policies_updated, policies_removed):
- """consolidate the new policies_updated, policies_removed to existing ones"""
- for policy_body in policies_updated:
- policy_name = policy_body.get(POLICY_NAME)
- policy = PolicyUtils.convert_to_policy(policy_body)
- if not policy:
- continue
- policy_id = policy.get(POLICY_ID)
-
- self._policies_updated[policy_id] = policy
-
- rm_policy_names = self._policies_removed.get(policy_id, {}).get(POLICY_NAMES)
- if rm_policy_names and policy_name in rm_policy_names:
- del rm_policy_names[policy_name]
-
- for policy_body in policies_removed:
- policy_name = policy_body.get(POLICY_NAME)
- policy = PolicyUtils.convert_to_policy(policy_body)
- if not policy:
- continue
- policy_id = policy.get(POLICY_ID)
-
- if policy_id in self._policies_removed:
- policy = self._policies_removed[policy_id]
-
- if POLICY_NAMES not in policy:
- policy[POLICY_NAMES] = {}
- policy[POLICY_NAMES][policy_name] = True
- self._policies_removed[policy_id] = policy
-
- req_message = ("policy-update notification - updated[{0}], removed[{1}]"
- .format(len(self._policies_updated),
- len(self._policies_removed)))
-
- if not self._audit:
- self._audit = Audit(job_name="policy_update",
- req_message=req_message,
- retry_get_config=True)
- else:
- self._audit.req_message = req_message
-
- self._logger.info(
- "pending(%s) for %s policies_updated %s policies_removed %s",
- self._audit.request_id, req_message,
- json.dumps(self._policies_updated), json.dumps(self._policies_removed))
-
+_LOGGER = Utils.get_logger(__file__)
class PolicyUpdater(Thread):
"""sequentially handle the policy-updates and catch-ups in its own policy_updater thread"""
- _logger = logging.getLogger("policy_handler.policy_updater")
-
def __init__(self, on_reconfigure_receiver):
"""init static config of PolicyUpdater."""
Thread.__init__(self, name="policy_updater", daemon=True)
@@ -132,7 +49,7 @@
self._aud_shutdown = None
self._aud_catch_up = None
self._aud_reconfigure = None
- self._policy_update = _PolicyUpdate()
+ self._policy_updates = pdp_client.PolicyUpdates()
self._catch_up_interval = None
self._reconfigure_interval = None
@@ -151,7 +68,7 @@
_, reconfigure = self._settings.get_by_key(Config.RECONFIGURE, {})
self._reconfigure_interval = reconfigure.get(Config.TIMER_INTERVAL) or 10*60
- PolicyUpdater._logger.info(
+ _LOGGER.info(
"intervals: catch_up(%s) reconfigure(%s): %s",
self._catch_up_interval, self._reconfigure_interval, self._settings)
self._settings.commit_change()
@@ -160,7 +77,7 @@
def policy_update(self, policies_updated, policies_removed):
"""enqueue the policy-updates"""
with self._lock:
- self._policy_update.push_policy_update(policies_updated, policies_removed)
+ self._policy_updates.push_policy_updates(policies_updated, policies_removed)
self._run.set()
def catch_up(self, audit=None):
@@ -168,7 +85,7 @@
with self._lock:
if not self._aud_catch_up:
self._aud_catch_up = audit or Audit(req_message=AUTO_CATCH_UP)
- PolicyUpdater._logger.info(
+ _LOGGER.info(
"catch_up %s request_id %s",
self._aud_catch_up.req_message, self._aud_catch_up.request_id
)
@@ -179,7 +96,7 @@
with self._lock:
if not self._aud_reconfigure:
self._aud_reconfigure = audit or Audit(req_message=AUTO_RECONFIGURE)
- PolicyUpdater._logger.info(
+ _LOGGER.info(
"%s request_id %s",
self._aud_reconfigure.req_message, self._aud_reconfigure.request_id
)
@@ -187,10 +104,10 @@
def run(self):
"""wait and run the policy-update in thread"""
- PolicyUpdater._logger.info("starting policy_updater...")
+ _LOGGER.info("starting policy_updater...")
self._run_reconfigure_timer()
while True:
- PolicyUpdater._logger.info("waiting for policy-updates...")
+ _LOGGER.info("waiting for policy-updates...")
self._run.wait()
with self._lock:
@@ -209,7 +126,7 @@
self._on_policy_update()
- PolicyUpdater._logger.info("exit policy-updater")
+ _LOGGER.info("exit policy-updater")
def _keep_running(self):
"""thread-safe check whether to continue running"""
@@ -226,18 +143,15 @@
return
if self._catch_up_timer:
- self._logger.info("next step catch_up_timer in %s", self._catch_up_interval)
+ _LOGGER.info("next step catch_up_timer in %s", self._catch_up_interval)
self._catch_up_timer.next(self._catch_up_interval)
return
self._catch_up_timer = StepTimer(
- "catch_up_timer",
- self._catch_up_interval,
- PolicyUpdater.catch_up,
- PolicyUpdater._logger,
- self
+ "catch_up_timer", self._catch_up_interval,
+ PolicyUpdater.catch_up, self
)
- self._logger.info("started catch_up_timer in %s", self._catch_up_interval)
+ _LOGGER.info("started catch_up_timer in %s", self._catch_up_interval)
self._catch_up_timer.start()
def _run_reconfigure_timer(self):
@@ -246,41 +160,38 @@
return
if self._reconfigure_timer:
- self._logger.info("next step reconfigure_timer in %s", self._reconfigure_interval)
+ _LOGGER.info("next step reconfigure_timer in %s", self._reconfigure_interval)
self._reconfigure_timer.next(self._reconfigure_interval)
return
self._reconfigure_timer = StepTimer(
- "reconfigure_timer",
- self._reconfigure_interval,
- PolicyUpdater.reconfigure,
- PolicyUpdater._logger,
- self
+ "reconfigure_timer", self._reconfigure_interval,
+ PolicyUpdater.reconfigure, self
)
- self._logger.info("started reconfigure_timer in %s", self._reconfigure_interval)
+ _LOGGER.info("started reconfigure_timer in %s", self._reconfigure_interval)
self._reconfigure_timer.start()
def _pause_catch_up_timer(self):
"""pause catch_up_timer"""
if self._catch_up_timer:
- self._logger.info("pause catch_up_timer")
+ _LOGGER.info("pause catch_up_timer")
self._catch_up_timer.pause()
def _stop_timers(self):
"""stop and destroy the catch_up and reconfigure timers"""
if self._catch_up_timer:
- self._logger.info("stopping catch_up_timer")
+ _LOGGER.info("stopping catch_up_timer")
self._catch_up_timer.stop()
self._catch_up_timer.join()
self._catch_up_timer = None
- self._logger.info("stopped catch_up_timer")
+ _LOGGER.info("stopped catch_up_timer")
if self._reconfigure_timer:
- self._logger.info("stopping reconfigure_timer")
+ _LOGGER.info("stopping reconfigure_timer")
self._reconfigure_timer.stop()
self._reconfigure_timer.join()
self._reconfigure_timer = None
- self._logger.info("stopped reconfigure_timer")
+ _LOGGER.info("stopped reconfigure_timer")
def _on_reconfigure(self):
"""bring the latest config and reconfigure"""
@@ -296,7 +207,7 @@
reconfigure_result = ""
try:
need_to_catch_up = False
- PolicyUpdater._logger.info(log_line)
+ _LOGGER.info(log_line)
active_prev = ServiceActivator.is_active_mode_of_operation(aud_reconfigure)
Config.discover(aud_reconfigure)
@@ -314,7 +225,7 @@
if self._set_timer_intervals():
changed_configs.append("timer_intervals")
- if PolicyRest.reconfigure():
+ if pdp_client.PolicyRest.reconfigure():
need_to_catch_up = True
changed_configs.append(Config.FIELD_POLICY_ENGINE)
@@ -335,7 +246,7 @@
Config.discovered_config.commit_change()
aud_reconfigure.audit_done(result=reconfigure_result)
- PolicyUpdater._logger.info(log_line + reconfigure_result)
+ _LOGGER.info(log_line + reconfigure_result)
if need_to_catch_up:
self._pause_catch_up_timer()
@@ -345,16 +256,15 @@
error_msg = "crash {} {}{}: {}: {}".format(
"_on_reconfigure", log_line, reconfigure_result, type(ex).__name__, str(ex))
- PolicyUpdater._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
aud_reconfigure.fatal(error_msg, error_code=AuditResponseCode.BUSINESS_PROCESS_ERROR)
aud_reconfigure.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
aud_reconfigure.audit_done(result=error_msg)
self._run_reconfigure_timer()
- PolicyUpdater._logger.info("policy_handler health: %s",
- json.dumps(aud_reconfigure.health(full=True)))
- PolicyUpdater._logger.info("process_info: %s", json.dumps(aud_reconfigure.process_info()))
+ _LOGGER.info("policy_handler health: %s", json.dumps(aud_reconfigure.health(full=True)))
+ _LOGGER.info("process_info: %s", json.dumps(aud_reconfigure.process_info()))
def _on_catch_up(self):
@@ -363,7 +273,7 @@
aud_catch_up = self._aud_catch_up
if self._aud_catch_up:
self._aud_catch_up = None
- self._policy_update.reset()
+ self._policy_updates.reset()
if not aud_catch_up:
return False
@@ -374,12 +284,11 @@
)
self._pause_catch_up_timer()
aud_catch_up.audit_done(result=catch_up_result)
- PolicyUpdater._logger.info(catch_up_result)
+ _LOGGER.info(catch_up_result)
self._run_catch_up_timer()
- PolicyUpdater._logger.info("policy_handler health: %s",
- json.dumps(aud_catch_up.health(full=True)))
- PolicyUpdater._logger.info("process_info: %s", json.dumps(aud_catch_up.process_info()))
+ _LOGGER.info("policy_handler health: %s", json.dumps(aud_catch_up.health(full=True)))
+ _LOGGER.info("process_info: %s", json.dumps(aud_catch_up.process_info()))
return False
log_line = "catch_up {0} request_id {1}".format(
@@ -388,25 +297,26 @@
catch_up_result = ""
try:
not_found_ok = None
- PolicyUpdater._logger.info(log_line)
+ _LOGGER.info(log_line)
self._pause_catch_up_timer()
- _, policies, policy_filters = PolicyMatcher.get_deployed_policies(aud_catch_up)
+ (_, policies,
+ policy_filters) = pdp_client.PolicyMatcher.get_deployed_policies(aud_catch_up)
catch_up_message = None
if aud_catch_up.is_not_found():
not_found_ok = True
else:
- _, catch_up_message = PolicyMatcher.build_catch_up_message(
+ _, catch_up_message = pdp_client.PolicyMatcher.build_catch_up_message(
aud_catch_up, policies, policy_filters)
if not_found_ok:
catch_up_result = ("- not sending catch-up "
"- no deployed policies or policy-filters")
- PolicyUpdater._logger.warning(catch_up_result)
+ _LOGGER.warning(catch_up_result)
elif not (catch_up_message and aud_catch_up.is_success()):
catch_up_result = "- not sending catch-up to deployment-handler due to errors"
- PolicyUpdater._logger.warning(catch_up_result)
+ _LOGGER.warning(catch_up_result)
elif catch_up_message.empty():
catch_up_result = "- not sending empty catch-up to deployment-handler"
else:
@@ -414,18 +324,18 @@
DeployHandler.policy_update(aud_catch_up, catch_up_message)
if not aud_catch_up.is_success():
catch_up_result = "- failed to send catch-up to deployment-handler"
- PolicyUpdater._logger.warning(catch_up_result)
+ _LOGGER.warning(catch_up_result)
else:
catch_up_result = "- sent catch-up to deployment-handler"
success, _, _ = aud_catch_up.audit_done(result=catch_up_result)
- PolicyUpdater._logger.info(log_line + " " + catch_up_result)
+ _LOGGER.info(log_line + " " + catch_up_result)
except Exception as ex:
error_msg = ("{0}: crash {1} {2} at {3}: {4}"
.format(aud_catch_up.request_id, type(ex).__name__, str(ex),
"on_catch_up", log_line + " " + catch_up_result))
- PolicyUpdater._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
aud_catch_up.fatal(error_msg, error_code=AuditResponseCode.BUSINESS_PROCESS_ERROR)
aud_catch_up.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
aud_catch_up.audit_done(result=error_msg)
@@ -433,9 +343,8 @@
self._run_catch_up_timer()
- PolicyUpdater._logger.info("policy_handler health: %s",
- json.dumps(aud_catch_up.health(full=True)))
- PolicyUpdater._logger.info("process_info: %s", json.dumps(aud_catch_up.process_info()))
+ _LOGGER.info("policy_handler health: %s", json.dumps(aud_catch_up.health(full=True)))
+ _LOGGER.info("process_info: %s", json.dumps(aud_catch_up.process_info()))
return success
@@ -443,42 +352,39 @@
"""handle the event of policy-updates"""
result = ""
with self._lock:
- audit, policies_updated, policies_removed = self._policy_update.pop_policy_update()
+ audit, policies_updated, policies_removed = self._policy_updates.pop_policy_updates()
if not audit:
return
log_line = "request_id: {} policies_updated: {} policies_removed: {}".format(
audit.request_id, json.dumps(policies_updated), json.dumps(policies_removed))
- PolicyUpdater._logger.info(log_line)
+ _LOGGER.info(log_line)
try:
not_found_ok = None
(updated_policies, removed_policies,
- policy_filter_matches) = PolicyMatcher.match_to_deployed_policies(
+ policy_filter_matches) = pdp_client.PolicyMatcher.match_to_deployed_policies(
audit, policies_updated, policies_removed)
if audit.is_not_found():
not_found_ok = True
elif updated_policies or removed_policies:
- updated_policies, removed_policies = PolicyRest.get_latest_updated_policies(
- (audit,
- [(policy_id, policy.get(POLICY_BODY, {}).get(POLICY_VERSION))
- for policy_id, policy in updated_policies.items()],
- [(policy_id, policy.get(POLICY_NAMES, {}))
- for policy_id, policy in removed_policies.items()]
- ))
+ (updated_policies,
+ removed_policies) = pdp_client.PolicyRest.get_latest_updated_policies(
+ audit, updated_policies, removed_policies
+ )
if not_found_ok:
result = ("- not sending policy-updates to deployment-handler "
"- no deployed policies or policy-filters")
- PolicyUpdater._logger.warning(result)
+ _LOGGER.warning(result)
elif not audit.is_success():
result = "- not sending policy-updates to deployment-handler due to errors"
- PolicyUpdater._logger.warning(result)
+ _LOGGER.warning(result)
elif not updated_policies and not removed_policies:
result = "- not sending empty policy-updates to deployment-handler"
- PolicyUpdater._logger.info(result)
+ _LOGGER.info(result)
else:
message = PolicyUpdateMessage(updated_policies, removed_policies,
policy_filter_matches, False)
@@ -493,19 +399,19 @@
log_line = "request_id[{}]: {}".format(audit.request_id, str(message))
if not audit.is_success():
result = "- failed to send to deployment-handler {}".format(log_updates)
- PolicyUpdater._logger.warning(result)
+ _LOGGER.warning(result)
else:
result = "- sent to deployment-handler {}".format(log_updates)
audit.audit_done(result=result)
- PolicyUpdater._logger.info(log_line + " " + result)
+ _LOGGER.info(log_line + " " + result)
except Exception as ex:
error_msg = ("{0}: crash {1} {2} at {3}: {4}"
.format(audit.request_id, type(ex).__name__, str(ex),
"on_policies_update", log_line + " " + result))
- PolicyUpdater._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
audit.fatal(error_msg, error_code=AuditResponseCode.BUSINESS_PROCESS_ERROR)
audit.set_http_status_code(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
@@ -517,7 +423,7 @@
def shutdown(self, audit):
"""Shutdown the policy-updater"""
- PolicyUpdater._logger.info("shutdown policy-updater")
+ _LOGGER.info("shutdown policy-updater")
with self._lock:
self._aud_shutdown = audit
self._run.set()
diff --git a/policyhandler/service_activator.py b/policyhandler/service_activator.py
index d51d11c..9c8a1b2 100644
--- a/policyhandler/service_activator.py
+++ b/policyhandler/service_activator.py
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""
ask service_activator for the mode_of_operation
@@ -30,23 +29,25 @@
"""
import json
-import logging
from copy import deepcopy
from urllib.parse import urljoin
import requests
from .config import Config, Settings
-from .onap.audit import REQUEST_X_ECOMP_REQUESTID, AuditHttpCode, Metrics
+from .onap.audit import (REQUEST_X_ECOMP_REQUESTID, Audit, AuditHttpCode,
+ Metrics)
from .policy_consts import TARGET_ENTITY
+from .utils import Utils
+_LOGGER = Utils.get_logger(__file__)
class ServiceActivator(object):
"""calling the service_activator web api to determine the mode_of_operation"""
- _logger = logging.getLogger("policy_handler.service_activator")
DEFAULT_TARGET_ENTITY = "service_activator"
DEFAULT_TIMEOUT_IN_SECS = 10
MODE_OF_OPERATION_ACTIVE = "active"
+ SERVICE_MODE = "service_mode"
_lazy_inited = False
_settings = Settings(Config.MODE_OF_OPERATION, Config.SERVICE_ACTIVATOR)
@@ -81,6 +82,7 @@
"""
ServiceActivator._custom_kwargs = {}
ServiceActivator._url = ServiceActivator._url_register = ""
+ Audit.register_item_health(ServiceActivator.SERVICE_MODE, ServiceActivator._get_service_mode)
try:
_, ServiceActivator._mode_of_operation = ServiceActivator._settings.get_by_key(
@@ -98,7 +100,7 @@
tls_ca_mode = config_sa.get(Config.TLS_CA_MODE)
ServiceActivator._custom_kwargs = Config.get_requests_kwargs(tls_ca_mode)
- ServiceActivator._logger.info(audit.info(
+ _LOGGER.info(audit.info(
"dns based routing to %s: url(%s) tls_ca_mode(%s) custom_kwargs(%s)",
ServiceActivator._target_entity, ServiceActivator._url_register,
tls_ca_mode, json.dumps(ServiceActivator._custom_kwargs)))
@@ -134,17 +136,31 @@
ServiceActivator._init(audit)
@staticmethod
- def is_active_mode_of_operation(audit):
+ def _get_service_mode():
+ """returns the service_mode as json to be reported by the healthcheck"""
+ return {
+ "is_active_mode_of_operation": ServiceActivator.is_active_mode_of_operation(),
+ "is_pdp_api_default": Config.is_pdp_api_default(log_status=False)
+ }
+
+ @staticmethod
+ def is_active_mode_of_operation(audit=None):
"""
mode_of_operation - whether the service is
active == True or passive == False
based on the current value of the mode_of_operation
+
+ temporary for R4 Dublin - passive for new PDP API
"""
active = (ServiceActivator._mode_of_operation is None
or ServiceActivator._mode_of_operation
== ServiceActivator.MODE_OF_OPERATION_ACTIVE)
- ServiceActivator._logger.info(audit.info(
- "mode_of_operation = {} active = {}".format(
+
+ if active and Config.is_pdp_api_default():
+ active = False
+
+ if audit:
+ _LOGGER.info(audit.info("mode_of_operation = {} active = {}".format(
ServiceActivator._mode_of_operation, active)))
return active
@@ -157,8 +173,7 @@
target_entity = ServiceActivator._target_entity
if not ServiceActivator._url:
- ServiceActivator._logger.info(audit.info(
- "no url found for {}".format(target_entity)))
+ _LOGGER.info(audit.info("no url found for {}".format(target_entity)))
return ServiceActivator.is_active_mode_of_operation(audit)
url = ServiceActivator._url_register
@@ -177,7 +192,7 @@
json.dumps(custom_kwargs))
log_line = log_action + " " + log_data
- ServiceActivator._logger.info(log_line)
+ _LOGGER.info(log_line)
metrics.metrics_start(log_line)
res = None
@@ -190,7 +205,7 @@
else AuditHttpCode.SERVER_INTERNAL_ERROR.value)
error_msg = "failed to {} {}: {} {}".format(
log_action, type(ex).__name__, str(ex), log_data)
- ServiceActivator._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
metrics.set_http_status_code(error_code)
audit.set_http_status_code(error_code)
metrics.metrics(error_msg)
@@ -204,7 +219,7 @@
metrics.metrics(log_line)
if res.status_code != requests.codes.ok:
- ServiceActivator._logger.error(log_line)
+ _LOGGER.error(log_line)
return ServiceActivator.is_active_mode_of_operation(audit)
result = res.json() or {}
diff --git a/policyhandler/step_timer.py b/policyhandler/step_timer.py
index 0f4f8e4..5ed4af5 100644
--- a/policyhandler/step_timer.py
+++ b/policyhandler/step_timer.py
@@ -1,5 +1,5 @@
# ================================================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""periodically callback"""
@@ -22,6 +21,9 @@
from datetime import datetime
from threading import Event, RLock, Thread
+from .utils import Utils
+
+_LOGGER = Utils.get_logger(__file__)
class StepTimer(Thread):
"""call on_time after interval number of seconds, then wait to continue"""
@@ -32,12 +34,11 @@
STATE_STOPPING = "stopping"
STATE_STOPPED = "stopped"
- def __init__(self, name, interval, on_time, logger, *args, **kwargs):
+ def __init__(self, name, interval, on_time, *args, **kwargs):
"""create step timer with controlled start. next step and pause"""
Thread.__init__(self, name=name)
self._interval = interval
self._on_time = on_time
- self._logger = logger
self._args = args
self._kwargs = kwargs
@@ -110,8 +111,8 @@
utcnow = datetime.utcnow()
self._req_time = (utcnow - self._req_ts).total_seconds()
self._req_ts = utcnow
- self._logger.info("{0}[{1}] {2}->{3}".format(
- self.name, self._req_time, prev_req, self.get_timer_status()))
+ _LOGGER.info("{}[{}] {}->{}".format(self.name, self._req_time, prev_req,
+ self.get_timer_status()))
def _log_substep(self, substep):
"""log timer substep"""
@@ -120,7 +121,8 @@
utcnow = datetime.utcnow()
self._substep_time = (utcnow - self._substep_ts).total_seconds()
self._substep_ts = utcnow
- self._logger.info("[{0}] {1}".format(self._substep_time, self.get_timer_status()))
+ _LOGGER.info("{}[{}] {}".format(self.name, self._substep_time,
+ self.get_timer_status()))
def _on_time_event(self):
"""execute the _on_time event"""
@@ -135,7 +137,7 @@
error_msg = ("{0}: crash {1} {2} at {3}: args({4}), kwargs({5})"
.format(self.name, type(ex).__name__, str(ex), "_on_time",
json.dumps(self._args), json.dumps(self._kwargs)))
- self._logger.exception(error_msg)
+ _LOGGER.exception(error_msg)
def run(self):
"""loop one step a time until stopped=finished"""
diff --git a/policyhandler/policy_utils.py b/policyhandler/utils.py
similarity index 70%
rename from policyhandler/policy_utils.py
rename to policyhandler/utils.py
index 08d26f0..d728e48 100644
--- a/policyhandler/policy_utils.py
+++ b/policyhandler/utils.py
@@ -1,5 +1,5 @@
# ================================================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,121 +14,44 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
-"""utils for policy usage and conversions"""
+"""utils and conversions"""
import json
import logging
-import re
+import os
from copy import deepcopy
from typing import Pattern
-from .policy_consts import (POLICY_BODY, POLICY_CONFIG, POLICY_ID, POLICY_NAME,
- POLICY_VERSION)
+class ToBeImplementedException(Exception):
+ """exception for to be implemented features of policy-handler"""
+ pass
-class PolicyUtils(object):
- """policy-client utils"""
- _logger = logging.getLogger("policy_handler.policy_utils")
- _policy_name_ext = re.compile('[.][0-9]+[.][a-zA-Z]+$')
-
- @staticmethod
- def extract_policy_id(policy_name):
- """ policy_name = policy_id + "." + <version> + "." + <extension>
- For instance,
- policy_name = DCAE_alex.Config_alex_policy_number_1.3.xml
- policy_id = DCAE_alex.Config_alex_policy_number_1
- policy_scope = DCAE_alex
- policy_class = Config
- policy_version = 3
- type = extension = xml
- delimiter = "."
- policy_class_delimiter = "_"
- policy_name in PAP = DCAE_alex.alex_policy_number_1
- """
- if not policy_name:
- return
- return PolicyUtils._policy_name_ext.sub('', policy_name)
-
- @staticmethod
- def parse_policy_config(policy):
- """try parsing the config in policy."""
- if not policy:
- return policy
- config = policy.get(POLICY_BODY, {}).get(POLICY_CONFIG)
- if config:
- policy[POLICY_BODY][POLICY_CONFIG] = Utils.safe_json_parse(config)
- return policy
-
- @staticmethod
- def convert_to_policy(policy_body):
- """wrap policy_body received from policy-engine with policy_id."""
- if not policy_body:
- return None
- policy_name = policy_body.get(POLICY_NAME)
- policy_version = policy_body.get(POLICY_VERSION)
- if not policy_name or not policy_version:
- return None
- policy_id = PolicyUtils.extract_policy_id(policy_name)
- if not policy_id:
- return None
- return {POLICY_ID:policy_id, POLICY_BODY:policy_body}
-
- @staticmethod
- def select_latest_policy(policy_bodies, expected_versions=None, ignore_policy_names=None):
- """For some reason, the policy-engine returns all version of the policy_bodies.
- DCAE-Controller is only interested in the latest version
- """
- if not policy_bodies:
- return
- latest_policy_body = {}
- for policy_body in policy_bodies:
- policy_name = policy_body.get(POLICY_NAME)
- policy_version = policy_body.get(POLICY_VERSION)
- if not policy_name or not policy_version or not policy_version.isdigit():
- continue
- if expected_versions and policy_version not in expected_versions:
- continue
- if ignore_policy_names and policy_name in ignore_policy_names:
- continue
-
- if (not latest_policy_body
- or int(latest_policy_body[POLICY_VERSION]) < int(policy_version)):
- latest_policy_body = policy_body
-
- return PolicyUtils.parse_policy_config(PolicyUtils.convert_to_policy(latest_policy_body))
-
- @staticmethod
- def select_latest_policies(policy_bodies):
- """For some reason, the policy-engine returns all version of the policy_bodies.
- DCAE-Controller is only interested in the latest versions
- """
- if not policy_bodies:
- return {}
- policies = {}
- for policy_body in policy_bodies:
- policy = PolicyUtils.convert_to_policy(policy_body)
- if not policy:
- continue
- policy_id = policy.get(POLICY_ID)
- policy_version = policy.get(POLICY_BODY, {}).get(POLICY_VERSION)
- if not policy_id or not policy_version or not policy_version.isdigit():
- continue
- if (policy_id not in policies
- or int(policy_version) > int(policies[policy_id][POLICY_BODY][POLICY_VERSION])):
- policies[policy_id] = policy
-
- for policy_id in policies:
- policies[policy_id] = PolicyUtils.parse_policy_config(policies[policy_id])
-
- return policies
-
class Utils(object):
"""general purpose utils"""
_logger = logging.getLogger("policy_handler.utils")
@staticmethod
+ def get_logger(file_path):
+ """get the logger for the file_path == __file__"""
+ logger_path = []
+ file_path = os.path.realpath(file_path)
+ logger_path.append(os.path.basename(file_path)[:-3])
+ while file_path:
+ file_path = os.path.dirname(file_path)
+ folder_name = os.path.basename(file_path)
+ if folder_name == "policyhandler" or len(logger_path) > 5:
+ break
+ if folder_name == "tests":
+ logger_path.append("unit_test")
+ break
+ logger_path.append(folder_name)
+
+ logger_path.append("policy_handler")
+ return logging.getLogger(".".join(reversed(logger_path)))
+
+ @staticmethod
def safe_json_parse(json_str):
"""try parsing json without exception - returns the json_str back if fails"""
if not json_str:
diff --git a/policyhandler/web_server.py b/policyhandler/web_server.py
index dc76353..dfd1b51 100644
--- a/policyhandler/web_server.py
+++ b/policyhandler/web_server.py
@@ -14,29 +14,27 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""web-server for policy_handler"""
import json
-import logging
from datetime import datetime
import cherrypy
+from . import pdp_client
from .config import Config
from .deploy_handler import PolicyUpdateMessage
from .onap.audit import Audit, AuditHttpCode
-from .policy_matcher import PolicyMatcher
from .policy_receiver import PolicyReceiver
-from .policy_rest import PolicyRest
+from .utils import Utils
class PolicyWeb(object):
"""run http API of policy-handler on 0.0.0.0:wservice_port - any incoming address"""
DATA_NOT_FOUND_ERROR = 404
HOST_INADDR_ANY = ".".join("0"*4)
- logger = logging.getLogger("policy_handler.policy_web")
+ logger = Utils.get_logger(__file__)
@staticmethod
def run_forever(audit):
@@ -84,7 +82,8 @@
PolicyWeb.logger.info("%s policy_id=%s headers=%s",
req_info, policy_id, json.dumps(cherrypy.request.headers))
- latest_policy = PolicyRest.get_latest_policy((audit, policy_id, None, None)) or {}
+ latest_policy = pdp_client.PolicyRest.get_latest_policy(
+ (audit, policy_id, None, None)) or {}
PolicyWeb.logger.info("res %s policy_id=%s latest_policy=%s",
req_info, policy_id, json.dumps(latest_policy))
@@ -104,9 +103,9 @@
PolicyWeb.logger.info("%s", req_info)
- result, policies, policy_filters = PolicyMatcher.get_deployed_policies(audit)
+ result, policies, policy_filters = pdp_client.PolicyMatcher.get_deployed_policies(audit)
if not result:
- result, policy_update = PolicyMatcher.build_catch_up_message(
+ result, policy_update = pdp_client.PolicyMatcher.build_catch_up_message(
audit, policies, policy_filters)
if policy_update and isinstance(policy_update, PolicyUpdateMessage):
result["policy_update"] = policy_update.get_message()
@@ -168,6 +167,9 @@
}
}
"""
+ if Config.is_pdp_api_default():
+ raise cherrypy.HTTPError(404, "temporarily unsupported due to the new pdp API")
+
if cherrypy.request.method == "GET":
return self._get_all_policies_latest()
@@ -184,7 +186,7 @@
PolicyWeb.logger.info("%s: policy_filter=%s headers=%s",
req_info, str_policy_filter, json.dumps(cherrypy.request.headers))
- result = PolicyRest.get_latest_policies(audit, policy_filter=policy_filter) or {}
+ result = pdp_client.PolicyRest.get_latest_policies(audit, policy_filter=policy_filter) or {}
result_str = json.dumps(result, sort_keys=True)
PolicyWeb.logger.info("result %s: policy_filter=%s result=%s",
diff --git a/pom.xml b/pom.xml
index 5b7f7a2..43f1421 100644
--- a/pom.xml
+++ b/pom.xml
@@ -16,7 +16,6 @@
limitations under the License.
============LICENSE_END=========================================================
-ECOMP is a trademark and service mark of AT&T Intellectual Property.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
@@ -30,7 +29,7 @@
<groupId>org.onap.dcaegen2.platform</groupId>
<artifactId>policy-handler</artifactId>
<name>dcaegen2-platform-policy-handler</name>
- <version>4.6.0-SNAPSHOT</version>
+ <version>5.0.0-SNAPSHOT</version>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
diff --git a/run_policy.sh b/run_policy.sh
index 72069f7..8055bea 100644
--- a/run_policy.sh
+++ b/run_policy.sh
@@ -1,7 +1,7 @@
#!/bin/bash
# ============LICENSE_START=======================================================
-# Copyright (c) 2017-2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2017-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -16,7 +16,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
mkdir -p logs
LOG_FILE=logs/policy_handler.log
@@ -24,9 +23,10 @@
echo "---------------------------------------------"
STARTED=$(date +%Y-%m-%d_%T.%N)
echo "${STARTED}: running ${BASH_SOURCE[0]}"
-echo "APP_VER =" $(python setup.py --version)
-echo "HOSTNAME =${HOSTNAME}"
-echo "CONSUL_URL =${CONSUL_URL}"
+echo "APP_VER="$(python setup.py --version)
+echo "HOSTNAME=${HOSTNAME}"
+echo "CONSUL_URL=${CONSUL_URL}"
+echo "PDP_API_VERSION=${PDP_API_VERSION}"
(pwd; uname -a; echo "/etc/hosts"; cat /etc/hosts; openssl version -a)
python -m policyhandler &
diff --git a/setup.py b/setup.py
index 3807bc6..eeef5c2 100644
--- a/setup.py
+++ b/setup.py
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""package for policy-handler of DCAE-Controller"""
@@ -23,7 +22,7 @@
setup(
name='policyhandler',
description='DCAE-Controller policy-handler to communicate with policy-engine',
- version="4.6.0",
+ version="5.0.0",
author='Alex Shatov',
packages=['policyhandler'],
zip_safe=False,
diff --git a/tests/conftest.py b/tests/conftest.py
index 0dcb2bb..c3f4100 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""
startdard pytest file that contains the shared fixtures
https://docs.pytest.org/en/latest/fixture.html
@@ -29,61 +28,61 @@
from policyhandler.deploy_handler import DeployHandler
from policyhandler.discovery import DiscoveryClient
from policyhandler.onap.audit import Audit
-from policyhandler.policy_consts import CATCH_UP, POLICY_NAME, TARGET_ENTITY
-from policyhandler.policy_receiver import PolicyReceiver
-from policyhandler.policy_rest import PolicyRest
+from policyhandler.policy_consts import CATCH_UP, TARGET_ENTITY
+from policyhandler.utils import Utils
from .mock_deploy_handler import MockDeploymentHandler
-from .mock_policy_engine import MockPolicyEngine
-from .mock_settings import Settings
+from .mock_settings import MockSettings
from .mock_tracker import MockHttpResponse, Tracker
-from .mock_websocket import MockWebSocket
+
+_LOGGER = Utils.get_logger(__file__)
+
+_LOGGER.info("init MockSettings")
+MockSettings.init()
+
+@pytest.fixture(scope="session", autouse=True)
+def _auto_setup__global():
+ """initialize the _auto_setup__global per the whole test session"""
+ _LOGGER.info("_auto_setup__global")
+
+ yield _auto_setup__global
+ Tracker.log_all_tests()
+ _LOGGER.info("teardown _auto_setup__global")
@pytest.fixture(autouse=True)
def _auto_test_cycle(request):
"""log all the test starts and ends"""
+ module_name = request.module.__name__.replace(".", "/")
if request.cls:
- test_name = "%s::%s::%s" % (request.module.__name__,
- request.cls.__name__,
- request.function.__name__)
+ test_name = "%s.py::%s::%s" % (module_name, request.cls.__name__,
+ request.function.__name__)
else:
- test_name = "%s::%s" % (request.module.__name__, request.function.__name__)
+ test_name = "%s.py::%s" % (module_name, request.function.__name__)
Tracker.reset(test_name)
- if Settings.logger:
- Settings.logger.info(">>>>>>> start %s", test_name)
+ _LOGGER.info("-"*75)
+ _LOGGER.info(">>>>>>> start [%s]: %s", len(Tracker.test_names), test_name)
yield _auto_test_cycle
- if Settings.logger:
- Settings.logger.info(">>>>>>> tracked messages: %s", Tracker.to_string())
- Settings.logger.info(">>>>>>> ended %s", test_name)
-
-
-@pytest.fixture(scope="session", autouse=True)
-def _auto_setup_policy_engine():
- """initialize the mock-policy-engine per the whole test session"""
- Settings.init()
-
- Settings.logger.info("create _auto_setup_policy_engine")
- MockPolicyEngine.init()
- yield _auto_setup_policy_engine
- Settings.logger.info("teardown _auto_setup_policy_engine")
+ _LOGGER.info(">>>>>>> tracked messages: %s", Tracker.to_string())
+ _LOGGER.info(">>>>>>> %s[%s]: %s", Tracker.get_status(test_name),
+ len(Tracker.test_names), test_name)
@pytest.fixture()
-def fix_pdp_post(monkeypatch):
- """monkeyed request /getConfig to PDP"""
- def monkeyed_policy_rest_post(uri, json=None, **kwargs):
- """monkeypatch for the POST to policy-engine"""
- res_json = MockPolicyEngine.get_config(json.get(POLICY_NAME))
- return MockHttpResponse("post", uri, res_json, json=json, **kwargs)
+def fix_cherrypy_engine_exit(monkeypatch):
+ """monkeyed cherrypy.engine.exit()"""
+ _LOGGER.info("setup fix_cherrypy_engine_exit")
- Settings.logger.info("setup fix_pdp_post")
- PolicyRest._lazy_init()
- monkeypatch.setattr('policyhandler.policy_rest.PolicyRest._requests_session.post',
- monkeyed_policy_rest_post)
- yield fix_pdp_post
- Settings.logger.info("teardown fix_pdp_post")
+ def monkeyed_cherrypy_engine_exit():
+ """monkeypatch for deploy_handler"""
+ _LOGGER.info("cherrypy_engine_exit()")
+
+ monkeypatch.setattr('policyhandler.web_server.cherrypy.engine.exit',
+ monkeyed_cherrypy_engine_exit)
+ yield fix_cherrypy_engine_exit
+ _LOGGER.info("teardown fix_cherrypy_engine_exit")
+
@pytest.fixture()
def fix_deploy_handler(monkeypatch):
@@ -98,7 +97,7 @@
return MockHttpResponse("get", uri, MockDeploymentHandler.get_deployed_policies(),
**kwargs)
- Settings.logger.info("setup fix_deploy_handler")
+ _LOGGER.info("setup fix_deploy_handler")
audit = None
if DeployHandler._lazy_inited is False:
audit = Audit(req_message="fix_deploy_handler")
@@ -112,79 +111,10 @@
yield fix_deploy_handler
if audit:
audit.audit_done("teardown")
- Settings.logger.info("teardown fix_deploy_handler")
+ _LOGGER.info("teardown fix_deploy_handler")
@pytest.fixture()
-def fix_cherrypy_engine_exit(monkeypatch):
- """monkeyed cherrypy.engine.exit()"""
- Settings.logger.info("setup fix_cherrypy_engine_exit")
-
- def monkeyed_cherrypy_engine_exit():
- """monkeypatch for deploy_handler"""
- Settings.logger.info("cherrypy_engine_exit()")
-
- monkeypatch.setattr('policyhandler.web_server.cherrypy.engine.exit',
- monkeyed_cherrypy_engine_exit)
- yield fix_cherrypy_engine_exit
- Settings.logger.info("teardown fix_cherrypy_engine_exit")
-
-
-@pytest.fixture()
-def fix_pdp_post_big(monkeypatch):
- """monkeyed request /getConfig to PDP"""
- def monkeyed_policy_rest_post(uri, **kwargs):
- """monkeypatch for the POST to policy-engine"""
- res_json = MockPolicyEngine.get_configs_all()
- return MockHttpResponse("post", uri, res_json, **kwargs)
-
- Settings.logger.info("setup fix_pdp_post_big")
- PolicyRest._lazy_init()
- monkeypatch.setattr('policyhandler.policy_rest.PolicyRest._requests_session.post',
- monkeyed_policy_rest_post)
- yield fix_pdp_post_big
- Settings.logger.info("teardown fix_pdp_post_big")
-
-
-class MockException(Exception):
- """mock exception"""
- pass
-
-
-@pytest.fixture()
-def fix_pdp_post_boom(monkeypatch):
- """monkeyed request /getConfig to PDP - exception"""
- def monkeyed_policy_rest_post_boom(uri, **_):
- """monkeypatch for the POST to policy-engine"""
- raise MockException("fix_pdp_post_boom {}".format(uri))
-
- Settings.logger.info("setup fix_pdp_post_boom")
- PolicyRest._lazy_init()
- monkeypatch.setattr('policyhandler.policy_rest.PolicyRest._requests_session.post',
- monkeyed_policy_rest_post_boom)
- yield fix_pdp_post_boom
- Settings.logger.info("teardown fix_pdp_post_boom")
-
-
-@pytest.fixture()
-def fix_select_latest_policies_boom(monkeypatch):
- """monkeyed exception"""
- def monkeyed_boom(*args, **kwargs):
- """monkeypatch for the select_latest_policies"""
- raise MockException("monkeyed_boom")
-
- Settings.logger.info("setup fix_select_latest_policies_boom")
- monkeypatch.setattr('policyhandler.policy_utils.PolicyUtils.select_latest_policies',
- monkeyed_boom)
- monkeypatch.setattr('policyhandler.policy_utils.PolicyUtils.select_latest_policy',
- monkeyed_boom)
- monkeypatch.setattr('policyhandler.policy_utils.PolicyUtils.extract_policy_id',
- monkeyed_boom)
-
- yield fix_select_latest_policies_boom
- Settings.logger.info("teardown fix_select_latest_policies_boom")
-
-@pytest.fixture()
def fix_discovery(monkeypatch):
"""monkeyed discovery request.get"""
def monkeyed_discovery(uri):
@@ -203,29 +133,28 @@
elif uri == DiscoveryClient.CONSUL_KV_MASK.format(
Config.consul_url, Config.system_name):
res_json = [{"Value": base64.b64encode(
- json.dumps(Settings.mock_config).encode()).decode("utf-8")}]
+ json.dumps(MockSettings.mock_config).encode()).decode("utf-8")}]
return MockHttpResponse("get", uri, res_json)
- Settings.logger.info("setup fix_discovery")
+ _LOGGER.info("setup fix_discovery")
monkeypatch.setattr('policyhandler.discovery.requests.get', monkeyed_discovery)
yield fix_discovery
- Settings.logger.info("teardown fix_discovery")
-
+ _LOGGER.info("teardown fix_discovery")
@pytest.fixture(scope="module")
def fix_auto_catch_up():
"""increase the frequency of auto catch_up"""
- Settings.logger.info("setup fix_auto_catch_up %s", json.dumps(Settings.mock_config))
- prev_config = copy.deepcopy(Settings.mock_config)
- Settings.mock_config.get(Config.SERVICE_NAME_POLICY_HANDLER, {}) \
+ _LOGGER.info("setup fix_auto_catch_up %s", json.dumps(MockSettings.mock_config))
+ prev_config = copy.deepcopy(MockSettings.mock_config)
+ MockSettings.mock_config.get(Config.SERVICE_NAME_POLICY_HANDLER, {}) \
.get(CATCH_UP, {})[Config.TIMER_INTERVAL] = 5
- Settings.logger.info("fix_auto_catch_up %s", json.dumps(Settings.mock_config))
- Settings.rediscover_config()
+ _LOGGER.info("fix_auto_catch_up %s", json.dumps(MockSettings.mock_config))
+ MockSettings.rediscover_config()
yield fix_auto_catch_up
- Settings.rediscover_config(prev_config)
- Settings.logger.info("teardown fix_auto_catch_up")
+ MockSettings.rediscover_config(prev_config)
+ _LOGGER.info("teardown fix_auto_catch_up")
@pytest.fixture()
@@ -235,7 +164,7 @@
"""monkeypatch for deploy_handler"""
return MockHttpResponse(
"put", uri,
- {"server_instance_uuid": Settings.deploy_handler_instance_uuid},
+ {"server_instance_uuid": MockSettings.deploy_handler_instance_uuid},
status_code=413, **kwargs
)
@@ -244,7 +173,7 @@
return MockHttpResponse("get", uri, MockDeploymentHandler.get_deployed_policies(),
**kwargs)
- Settings.logger.info("setup fix_deploy_handler_413")
+ _LOGGER.info("setup fix_deploy_handler_413")
audit = None
if DeployHandler._lazy_inited is False:
audit = Audit(req_message="fix_deploy_handler_413")
@@ -258,7 +187,7 @@
yield fix_deploy_handler_413
if audit:
audit.audit_done("teardown")
- Settings.logger.info("teardown fix_deploy_handler_413")
+ _LOGGER.info("teardown fix_deploy_handler_413")
@pytest.fixture()
@@ -274,7 +203,7 @@
return MockHttpResponse("get", uri, MockDeploymentHandler.default_response(),
**kwargs)
- Settings.logger.info("setup fix_deploy_handler_404")
+ _LOGGER.info("setup fix_deploy_handler_404")
audit = None
if DeployHandler._lazy_inited is False:
audit = Audit(req_message="fix_deploy_handler_404")
@@ -288,12 +217,4 @@
yield fix_deploy_handler_404
if audit:
audit.audit_done("teardown")
- Settings.logger.info("teardown fix_deploy_handler_404")
-
-@pytest.fixture()
-def fix_policy_receiver_websocket(monkeypatch):
- """monkeyed websocket for policy_receiver"""
- Settings.logger.info("setup fix_policy_receiver_websocket")
- monkeypatch.setattr('policyhandler.policy_receiver.websocket', MockWebSocket)
- yield fix_policy_receiver_websocket
- Settings.logger.info("teardown fix_policy_receiver_websocket")
+ _LOGGER.info("teardown fix_deploy_handler_404")
diff --git a/tests/etc_config.json b/tests/etc_config.json
new file mode 100644
index 0000000..d7715b2
--- /dev/null
+++ b/tests/etc_config.json
@@ -0,0 +1,47 @@
+{
+ "wservice_port" : 25577,
+ "consul_url" : "http://unit-test-consul:850000",
+ "consul_timeout_in_secs" : 60,
+ "pdp_api_version" : null,
+ "policy_handler" : {
+ "system" : "policy_handler",
+ "tls" : {
+ "cert_directory" : "etc/tls/certs/",
+ "cacert" : "att_internal_root.crt",
+ "private_key" : "key.pem",
+ "server_cert" : "cert.pem",
+ "server_ca_chain" : "ca_chain.pem"
+ }
+ },
+ "logging" : {
+ "version": 1,
+ "disable_existing_loggers": false,
+ "formatters": {
+ "local": {
+ "format": "%(asctime)s.%(msecs)03d %(levelname)+8s %(threadName)s %(name)s.%(funcName)s: %(message)s",
+ "datefmt": "%Y%m%d_%H%M%S"
+ }
+ },
+ "handlers": {
+ "file": {
+ "class": "logging.handlers.TimedRotatingFileHandler",
+ "formatter": "local",
+ "filename" : "logs/policy_handler.log",
+ "level": "DEBUG",
+ "when": "midnight",
+ "interval": 1,
+ "backupCount": 10,
+ "delay": true
+ }
+ },
+ "loggers": {
+ "policy_handler" : {
+ "handlers": ["file"],
+ "propagate":false
+ }
+ },
+ "root": {
+ "handlers": ["file"]
+ }
+ }
+}
\ No newline at end of file
diff --git a/tests/main/__init__.py b/tests/main/__init__.py
new file mode 100644
index 0000000..b8e4605
--- /dev/null
+++ b/tests/main/__init__.py
@@ -0,0 +1,20 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+
+# empty __init__.py so that pytest can add correct path to coverage report, -- per pytest
+# best practice guideline
diff --git a/tests/main/conftest.py b/tests/main/conftest.py
new file mode 100644
index 0000000..c985380
--- /dev/null
+++ b/tests/main/conftest.py
@@ -0,0 +1,56 @@
+# ============LICENSE_START=======================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+"""
+startdard pytest file that contains the shared fixtures
+https://docs.pytest.org/en/latest/fixture.html
+"""
+
+import pytest
+
+from policyhandler import pdp_client
+from policyhandler.pdp_api.pdp_consts import PDP_POLICY_ID, PDP_REQ_RESOURCE
+from policyhandler.utils import Utils
+
+from ..mock_tracker import MockHttpResponse
+from .mock_policy_engine import MockPolicyEngine
+
+_LOGGER = Utils.get_logger(__file__)
+
+@pytest.fixture(scope="session", autouse=True)
+def _auto_setup_policy_engine():
+ """initialize the mock-policy-engine per the whole test session"""
+ _LOGGER.info("create _auto_setup_policy_engine")
+ MockPolicyEngine.init()
+ yield _auto_setup_policy_engine
+ _LOGGER.info("teardown _auto_setup_policy_engine")
+
+@pytest.fixture()
+def fix_pdp_post(monkeypatch):
+ """monkeyed request /decision/v1 to PDP"""
+ def monkeyed_policy_rest_post(uri, json=None, **kwargs):
+ """monkeypatch for the POST to policy-engine"""
+ policy_ids = json.get(PDP_REQ_RESOURCE, {}).get(PDP_POLICY_ID)
+ policy_id = policy_ids and policy_ids[0]
+ res_json = MockPolicyEngine.get_policy(policy_id)
+ return MockHttpResponse("post", uri, res_json, json=json, **kwargs)
+
+ _LOGGER.info("setup fix_pdp_post")
+ pdp_client.PolicyRest._lazy_init()
+ monkeypatch.setattr('policyhandler.pdp_client.PolicyRest._requests_session.post',
+ monkeyed_policy_rest_post)
+ yield fix_pdp_post
+ _LOGGER.info("teardown fix_pdp_post")
diff --git a/tests/main/mock_expected.py b/tests/main/mock_expected.py
new file mode 100644
index 0000000..450394c
--- /dev/null
+++ b/tests/main/mock_expected.py
@@ -0,0 +1,526 @@
+# ============LICENSE_START=======================================================
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+"""expected message history per test for pdp API 2019 and after"""
+
+
+HISTORY_EXPECTED = {
+ "tests/main/test_policy_rest.py::test_get_policy_latest" : [
+ {
+ "request": {
+ "headers": {
+ "Accept": "application/json",
+ "Authorization": "Basic auth",
+ "ClientAuth": "Basic user",
+ "Content-Type": "application/json",
+ "Environment": "TEST",
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "ONAPComponent": "policy_handler",
+ "ONAPInstance": "*",
+ "ONAPName": "DCAE",
+ "action": "configure",
+ "resource": {
+ "policy-id": [
+ "test_scope_prefix.pdp_desition_sit"
+ ]
+ }
+ },
+ "method": "post",
+ "params": None,
+ "uri": "https://unit-test-pdp-server:8081000/decision/v1/"
+ },
+ "res": "*",
+ "status_code": 200
+ }
+ ],
+ "tests/main/test_policyhandler.py::WebServerTest::test_web_all_policies_latest": [
+ {
+ "request": {
+ "headers": {
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": None,
+ "method": "get",
+ "params": {
+ "cfy_tenant_name": "default_tenant"
+ },
+ "uri": "http://unit-test-deployment_handler:8188000/policy"
+ },
+ "res": "*",
+ "status_code": 200
+ },
+ {
+ "request": {
+ "headers": {
+ "Accept": "application/json",
+ "Authorization": "Basic auth",
+ "ClientAuth": "Basic user",
+ "Content-Type": "application/json",
+ "Environment": "TEST",
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "policyName": "test_scope_prefix.Config_.*"
+ },
+ "method": "post",
+ "params": None,
+ "uri": "https://unit-test-pdp-server:8081000/pdp/api/getConfig"
+ },
+ "res": "*",
+ "status_code": 200
+ }
+ ],
+ "tests/main/test_policyhandler.py::WebServerTest::test_web_policies_latest": [
+ {
+ "request": {
+ "headers": {
+ "Accept": "application/json",
+ "Authorization": "Basic auth",
+ "ClientAuth": "Basic user",
+ "Content-Type": "application/json",
+ "Environment": "TEST",
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "policyName": "test_scope_prefix.Config_amet.*"
+ },
+ "method": "post",
+ "params": None,
+ "uri": "https://unit-test-pdp-server:8081000/pdp/api/getConfig"
+ },
+ "res": "*",
+ "status_code": 200
+ }
+ ],
+ "tests/main/test_policyhandler.py::WebServerTest::test_web_policy_latest": [
+ {
+ "request": {
+ "headers": {
+ "Accept": "application/json",
+ "Authorization": "Basic auth",
+ "ClientAuth": "Basic user",
+ "Content-Type": "application/json",
+ "Environment": "TEST",
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "ONAPComponent": "policy_handler",
+ "ONAPInstance": "*",
+ "ONAPName": "DCAE",
+ "action": "configure",
+ "resource": {
+ "policy-id": [
+ "test_scope_prefix.pdp_desition_sit"
+ ]
+ }
+ },
+ "method": "post",
+ "params": None,
+ "uri": "https://unit-test-pdp-server:8081000/decision/v1/"
+ },
+ "res": "*",
+ "status_code": 200
+ }
+ ],
+ "tests/main/test_policyhandler.py::WebServerTest::test_zzzzz_shutdown": [
+ {
+ "request": {
+ "headers": {
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": None,
+ "method": "get",
+ "params": {
+ "cfy_tenant_name": "default_tenant"
+ },
+ "uri": "http://unit-test-deployment_handler:8188000/policy"
+ },
+ "res": "*",
+ "status_code": 200
+ },
+ {
+ "request": {
+ "headers": {
+ "Accept": "application/json",
+ "Authorization": "Basic auth",
+ "ClientAuth": "Basic user",
+ "Content-Type": "application/json",
+ "Environment": "TEST",
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "policyName": "test_scope_prefix.Config_.*"
+ },
+ "method": "post",
+ "params": None,
+ "uri": "https://unit-test-pdp-server:8081000/pdp/api/getConfig"
+ },
+ "res": "*",
+ "status_code": 200
+ },
+ {
+ "request": {
+ "headers": {
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "catch_up": True,
+ "latest_policies": {
+ "test_scope_prefix.Config_Lorem": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "0",
+ "policy_updated_to_ver": "1",
+ "updated_policy_id": "test_scope_prefix.Config_Lorem"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_Lorem.1.xml",
+ "policyVersion": "1",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_Lorem"
+ },
+ "test_scope_prefix.Config_amet": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "4",
+ "policy_updated_to_ver": "5",
+ "updated_policy_id": "test_scope_prefix.Config_amet"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_amet.5.xml",
+ "policyVersion": "5",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_amet"
+ },
+ "test_scope_prefix.Config_ametist": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "6",
+ "policy_updated_to_ver": "7",
+ "updated_policy_id": "test_scope_prefix.Config_ametist"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_ametist.7.xml",
+ "policyVersion": "7",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_ametist"
+ },
+ "test_scope_prefix.Config_consectetur": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "5",
+ "policy_updated_to_ver": "6",
+ "updated_policy_id": "test_scope_prefix.Config_consectetur"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_consectetur.6.xml",
+ "policyVersion": "6",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_consectetur"
+ },
+ "test_scope_prefix.Config_dolor": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "2",
+ "policy_updated_to_ver": "3",
+ "updated_policy_id": "test_scope_prefix.Config_dolor"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_dolor.3.xml",
+ "policyVersion": "3",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_dolor"
+ },
+ "test_scope_prefix.Config_ipsum": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "1",
+ "policy_updated_to_ver": "2",
+ "updated_policy_id": "test_scope_prefix.Config_ipsum"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_ipsum.2.xml",
+ "policyVersion": "2",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_ipsum"
+ },
+ "test_scope_prefix.Config_sit": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "3",
+ "policy_updated_to_ver": "4",
+ "updated_policy_id": "test_scope_prefix.Config_sit"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_sit.4.xml",
+ "policyVersion": "4",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_sit"
+ }
+ },
+ "policy_filter_matches": {
+ "test_scope_prefix.Config_Lorem": {},
+ "test_scope_prefix.Config_amet": {},
+ "test_scope_prefix.Config_ametist": {},
+ "test_scope_prefix.Config_consectetur": {},
+ "test_scope_prefix.Config_dolor": {},
+ "test_scope_prefix.Config_ipsum": {},
+ "test_scope_prefix.Config_sit": {}
+ },
+ "removed_policies": {}
+ },
+ "method": "put",
+ "params": {
+ "cfy_tenant_name": "default_tenant"
+ },
+ "uri": "http://unit-test-deployment_handler:8188000/policy"
+ },
+ "res": "*",
+ "status_code": 200
+ },
+ {
+ "request": {
+ "headers": {
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": None,
+ "method": "get",
+ "params": {
+ "cfy_tenant_name": "default_tenant"
+ },
+ "uri": "http://unit-test-deployment_handler:8188000/policy"
+ },
+ "res": "*",
+ "status_code": 200
+ },
+ {
+ "request": {
+ "headers": {
+ "Accept": "application/json",
+ "Authorization": "Basic auth",
+ "ClientAuth": "Basic user",
+ "Content-Type": "application/json",
+ "Environment": "TEST",
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "policyName": "test_scope_prefix.Config_ipsum"
+ },
+ "method": "post",
+ "params": None,
+ "uri": "https://unit-test-pdp-server:8081000/pdp/api/getConfig"
+ },
+ "res": "*",
+ "status_code": 200
+ },
+ {
+ "request": {
+ "headers": {
+ "Accept": "application/json",
+ "Authorization": "Basic auth",
+ "ClientAuth": "Basic user",
+ "Content-Type": "application/json",
+ "Environment": "TEST",
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "policyName": "test_scope_prefix.Config_sit"
+ },
+ "method": "post",
+ "params": None,
+ "uri": "https://unit-test-pdp-server:8081000/pdp/api/getConfig"
+ },
+ "res": "*",
+ "status_code": 200
+ },
+ {
+ "request": {
+ "headers": {
+ "Accept": "application/json",
+ "Authorization": "Basic auth",
+ "ClientAuth": "Basic user",
+ "Content-Type": "application/json",
+ "Environment": "TEST",
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "policyName": "test_scope_prefix.Config_consectetur"
+ },
+ "method": "post",
+ "params": None,
+ "uri": "https://unit-test-pdp-server:8081000/pdp/api/getConfig"
+ },
+ "res": "*",
+ "status_code": 200
+ },
+ {
+ "request": {
+ "headers": {
+ "X-ECOMP-RequestID": "*"
+ },
+ "json": {
+ "catch_up": False,
+ "latest_policies": {
+ "test_scope_prefix.Config_consectetur": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "5",
+ "policy_updated_to_ver": "6",
+ "updated_policy_id": "test_scope_prefix.Config_consectetur"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_consectetur.6.xml",
+ "policyVersion": "6",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_consectetur"
+ },
+ "test_scope_prefix.Config_ipsum": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "1",
+ "policy_updated_to_ver": "2",
+ "updated_policy_id": "test_scope_prefix.Config_ipsum"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_ipsum.2.xml",
+ "policyVersion": "2",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_ipsum"
+ },
+ "test_scope_prefix.Config_sit": {
+ "policy_body": {
+ "config": {
+ "policy_hello": "world!",
+ "policy_updated_from_ver": "3",
+ "policy_updated_to_ver": "4",
+ "updated_policy_id": "test_scope_prefix.Config_sit"
+ },
+ "matchingConditions": {
+ "ConfigName": "alex_config_name",
+ "ONAPName": "DCAE"
+ },
+ "policyConfigMessage": "Config Retrieved! ",
+ "policyConfigStatus": "CONFIG_RETRIEVED",
+ "policyName": "test_scope_prefix.Config_sit.4.xml",
+ "policyVersion": "4",
+ "property": None,
+ "responseAttributes": {},
+ "type": "JSON"
+ },
+ "policy_id": "test_scope_prefix.Config_sit"
+ }
+ },
+ "policy_filter_matches": {
+ "test_scope_prefix.Config_consectetur": {},
+ "test_scope_prefix.Config_ipsum": {},
+ "test_scope_prefix.Config_sit": {}
+ },
+ "removed_policies": {}
+ },
+ "method": "put",
+ "params": {
+ "cfy_tenant_name": "default_tenant"
+ },
+ "uri": "http://unit-test-deployment_handler:8188000/policy"
+ },
+ "res": "*",
+ "status_code": 200
+ }
+ ]
+}
diff --git a/tests/main/mock_policy_engine.py b/tests/main/mock_policy_engine.py
new file mode 100644
index 0000000..1bac045
--- /dev/null
+++ b/tests/main/mock_policy_engine.py
@@ -0,0 +1,96 @@
+# ============LICENSE_START=======================================================
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+"""mocking for the policy-engine - shared by many tests"""
+
+import copy
+import json
+
+from policyhandler.pdp_api.pdp_consts import (PDP_METADATA,
+ PDP_POLICY_ID,
+ PDP_POLICY_VERSION,
+ PDP_POLICIES, PDP_PROPERTIES)
+from policyhandler.pdp_api.policy_utils import PolicyUtils
+from policyhandler.utils import Utils
+
+_LOGGER = Utils.get_logger(__file__)
+
+
+class MockPolicyEngine(object):
+ """pretend this is the policy-engine"""
+ scope_prefix = "test_scope_prefix.pdp_desition_"
+ LOREM_IPSUM = """Lorem ipsum dolor sit amet consectetur ametist""".split()
+ LONG_TEXT = "0123456789" * 100
+ _policies = {}
+
+ _inited = False
+
+ @staticmethod
+ def init():
+ """init collection of policies: policy_version = policy_index + 1"""
+ if MockPolicyEngine._inited:
+ return
+ MockPolicyEngine._inited = True
+
+ MockPolicyEngine._policies = dict(
+ (policy_id, MockPolicyEngine._create_policy_body(policy_id, policy_version))
+ for policy_id, policy_version in
+ [(MockPolicyEngine.get_policy_id(policy_index), policy_index + 1)
+ for policy_index in range(1 + len(MockPolicyEngine.LOREM_IPSUM))]
+ )
+ _LOGGER.info("_policies: %s", json.dumps(MockPolicyEngine._policies))
+
+ @staticmethod
+ def get_policy_id(policy_index):
+ """get the policy_id by policy_index"""
+ return (MockPolicyEngine.scope_prefix
+ + MockPolicyEngine.LOREM_IPSUM[
+ policy_index % len(MockPolicyEngine.LOREM_IPSUM)])
+
+ @staticmethod
+ def get_policy(policy_id):
+ """find policy the way the policy-engine finds"""
+ if policy_id not in MockPolicyEngine._policies:
+ return {}
+ return {PDP_POLICIES: {policy_id: copy.deepcopy(MockPolicyEngine._policies[policy_id])}}
+
+ @staticmethod
+ def gen_policy_latest(policy_index, version_offset=0):
+ """generate the policy response from policy-handler by policy_index = version - 1"""
+ policy_id = MockPolicyEngine.get_policy_id(policy_index)
+ policy = PolicyUtils.convert_to_policy(
+ MockPolicyEngine._create_policy_body(policy_id, policy_index + 1 - version_offset)
+ )
+ return policy_id, policy
+
+ @staticmethod
+ def _create_policy_body(policy_id, policy_version=1):
+ """returns a fake policy-body"""
+ return {
+ "type": "unit.test.type.policies",
+ "version": "1.0.0",
+ PDP_METADATA: {
+ PDP_POLICY_ID: policy_id,
+ PDP_POLICY_VERSION: policy_version,
+ "description": "description for {}".format(policy_id)
+ },
+ PDP_PROPERTIES: {
+ "policy_updated_from_ver": (policy_version - 1),
+ "policy_updated_to_ver": policy_version,
+ "policy_hello": "world!",
+ "updated_policy_id": policy_id
+ }
+ }
diff --git a/tests/main/pdp_policies.json b/tests/main/pdp_policies.json
new file mode 100644
index 0000000..b7d0d9e
--- /dev/null
+++ b/tests/main/pdp_policies.json
@@ -0,0 +1,25 @@
+{
+ "policies": {
+ "onap.scaleout.tca": {
+ "type": "onap.policies.monitoring.cdap.tca.hi.lo.app",
+ "version": "1.0.0",
+ "metadata": {
+ "policy-id": "onap.scaleout.tca",
+ "policy-version": 1,
+ "description": "The scaleout policy for vDNS"
+ },
+ "properties": {
+ "tca_policy": {
+ "domain": "measurementsForVfScaling",
+ "metricsPerEventName": [
+ {
+ "eventName": "vLoadBalancer",
+ "controlLoopSchemaType": "VNF",
+ "policyScope": "type=configuration"
+ }
+ ]
+ }
+ }
+ }
+ }
+}
\ No newline at end of file
diff --git a/tests/test_policy_rest.py b/tests/main/test_policy_rest.py
similarity index 72%
rename from tests/test_policy_rest.py
rename to tests/main/test_policy_rest.py
index ac2529a..964a21d 100644
--- a/tests/test_policy_rest.py
+++ b/tests/main/test_policy_rest.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,21 +14,20 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test policy_rest methods directly"""
import json
import pytest
+from policyhandler import pdp_client
from policyhandler.onap.audit import Audit
-from policyhandler.policy_rest import PolicyRest
-from policyhandler.policy_utils import Utils
+from policyhandler.utils import Utils
+from ..mock_tracker import Tracker
from .mock_policy_engine import MockPolicyEngine
-from .mock_settings import Settings
-from .mock_tracker import Tracker
+_LOGGER = Utils.get_logger(__file__)
@pytest.mark.usefixtures("fix_pdp_post")
def test_get_policy_latest():
@@ -37,11 +36,12 @@
audit = Audit(job_name="test_get_policy_latest",
req_message="get /policy_latest/{}".format(policy_id or ""))
- policy_latest = PolicyRest.get_latest_policy((audit, policy_id, None, None)) or {}
+
+ policy_latest = pdp_client.PolicyRest.get_latest_policy((audit, policy_id, None, None)) or {}
audit.audit_done(result=json.dumps(policy_latest))
- Settings.logger.info("expected_policy: %s", json.dumps(expected_policy))
- Settings.logger.info("policy_latest: %s", json.dumps(policy_latest))
+ _LOGGER.info("expected_policy: %s", json.dumps(expected_policy))
+ _LOGGER.info("policy_latest: %s", json.dumps(policy_latest))
assert Utils.are_the_same(policy_latest, expected_policy)
Tracker.validate()
diff --git a/tests/main/test_policyhandler.py b/tests/main/test_policyhandler.py
new file mode 100644
index 0000000..73ab9ca
--- /dev/null
+++ b/tests/main/test_policyhandler.py
@@ -0,0 +1,130 @@
+# ============LICENSE_START=======================================================
+# Copyright (c) 2017-2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+"""test of the package for policy-handler of DCAE-Controller"""
+
+import json
+import time
+import uuid
+
+import cherrypy
+import pytest
+from cherrypy.test.helper import CPWebCase
+
+from policyhandler.config import Config
+from policyhandler.onap.audit import REQUEST_X_ECOMP_REQUESTID, Audit
+from policyhandler.pdp_api.pdp_consts import POLICY_NAME
+from policyhandler.policy_consts import LATEST_POLICIES
+from policyhandler.policy_receiver import PolicyReceiver
+from policyhandler.utils import Utils
+from policyhandler.web_server import _PolicyWeb
+
+from ..mock_tracker import Tracker
+from .mock_policy_engine import MockPolicyEngine
+
+_LOGGER = Utils.get_logger(__file__)
+
+
+@pytest.mark.usefixtures("fix_pdp_post", "fix_discovery")
+class WebServerTest(CPWebCase):
+ """testing the web-server - runs tests in alphabetical order of method names"""
+ def setup_server():
+ """setup the web-server"""
+ cherrypy.tree.mount(_PolicyWeb(), '/')
+
+ setup_server = staticmethod(setup_server)
+
+ def test_web_healthcheck(self):
+ """test /healthcheck"""
+ result = self.getPage("/healthcheck")
+ _LOGGER.info("healthcheck result: %s", result)
+ _LOGGER.info("got healthcheck: %s", self.body)
+ self.assertStatus('200 OK')
+
+ Tracker.validate()
+
+ def test_web_policy_latest(self):
+ """test /policy_latest/<policy-id>"""
+ policy_id, expected_policy = MockPolicyEngine.gen_policy_latest(3)
+
+ self.getPage("/policy_latest/{0}".format(policy_id or ""))
+ self.assertStatus('200 OK')
+
+ policy_latest = json.loads(self.body)
+
+ _LOGGER.info("policy_latest: %s", self.body)
+ _LOGGER.info("expected_policy: %s", json.dumps(expected_policy))
+ assert Utils.are_the_same(policy_latest, expected_policy)
+
+ result = self.getPage("/healthcheck")
+ _LOGGER.info("healthcheck result: %s", result)
+
+ Tracker.validate()
+
+ @pytest.mark.usefixtures("fix_deploy_handler")
+ def test_web_all_policies_latest(self):
+ """test GET /policies_latest"""
+
+ result = self.getPage("/policies_latest")
+ _LOGGER.info("result: %s", result)
+ _LOGGER.info("body: %s", self.body)
+
+ self.assertStatus('404 Not Found')
+
+ def test_web_policies_latest(self):
+ """test POST /policies_latest with policyName"""
+ body = json.dumps({"junk": "to-be-developed"})
+ result = self.getPage("/policies_latest", method='POST',
+ body=body,
+ headers=[
+ (REQUEST_X_ECOMP_REQUESTID, str(uuid.uuid4())),
+ ("Content-Type", "application/json"),
+ ('Content-Length', str(len(body)))
+ ])
+ _LOGGER.info("result: %s", result)
+ _LOGGER.info("body: %s", self.body)
+
+ self.assertStatus('404 Not Found')
+
+ @pytest.mark.usefixtures(
+ "fix_deploy_handler",
+ "fix_cherrypy_engine_exit")
+ def test_zzzzz_shutdown(self):
+ """test shutdown"""
+ _LOGGER.info("start shutdown")
+ assert not PolicyReceiver.is_running()
+ audit = Audit(job_name="test_zzzzz_shutdown", req_message="start shutdown")
+ PolicyReceiver.run(audit)
+
+ result = self.getPage("/healthcheck")
+ _LOGGER.info("healthcheck result: %s", result)
+
+ WebServerTest.do_gc_test = False
+ _LOGGER.info("shutdown...")
+ audit.audit_done("shutdown")
+ result = self.getPage("/shutdown")
+ _LOGGER.info("shutdown result: %s", result)
+ self.assertStatus('200 OK')
+ _LOGGER.info("got shutdown: %s", self.body)
+ time.sleep(1)
+ assert not PolicyReceiver.is_running()
+
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ Tracker.validate()
diff --git a/tests/mock_config.json b/tests/mock_config.json
index ef02a15..30e3251 100644
--- a/tests/mock_config.json
+++ b/tests/mock_config.json
@@ -13,6 +13,7 @@
},
"policy_engine" : {
"url" : "https://unit-test-pdp-server:8081000",
+ "path_decision" : "/decision/v1/",
"path_notifications" : "/pdp/notifications",
"path_api" : "/pdp/api/",
"headers" : {
diff --git a/tests/mock_deploy_handler.py b/tests/mock_deploy_handler.py
index ebbbfc7..cd8e791 100644
--- a/tests/mock_deploy_handler.py
+++ b/tests/mock_deploy_handler.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,14 +14,13 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""mocking for the deployment-handler - shared by many tests"""
-from policyhandler.policy_consts import (POLICY_BODY, POLICY_ID,
- POLICY_VERSION, POLICY_VERSIONS)
+from policyhandler.pdp_api_v0.pdp_consts import POLICY_VERSION
+from policyhandler.policy_consts import POLICY_BODY, POLICY_ID, POLICY_VERSIONS
-from .mock_policy_engine import MockPolicyEngine
-from .mock_settings import Settings
+from .mock_settings import MockSettings
+from .pdp_api_v0.mock_policy_engine import MockPolicyEngine2018
class MockDeploymentHandler(object):
@@ -30,7 +29,7 @@
@staticmethod
def default_response():
"""generate the deployed policies message"""
- return {"server_instance_uuid": Settings.deploy_handler_instance_uuid}
+ return {"server_instance_uuid": MockSettings.deploy_handler_instance_uuid}
@staticmethod
def get_deployed_policies():
@@ -42,7 +41,7 @@
POLICY_VERSIONS: {policy.get(POLICY_BODY, {}).get(POLICY_VERSION, "999"): True},
"pending_update": False})
for policy_id, policy in (
- MockPolicyEngine.gen_all_policies_latest(version_offset=1).items()))
+ MockPolicyEngine2018.gen_all_policies_latest(version_offset=1).items()))
response["policies"] = policies
return response
diff --git a/tests/mock_settings.py b/tests/mock_settings.py
index 8dec8e5..9e99561 100644
--- a/tests/mock_settings.py
+++ b/tests/mock_settings.py
@@ -14,12 +14,12 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""settings that are general to all tests"""
import copy
+import importlib
import json
-import logging
+import os
import sys
import uuid
from functools import wraps
@@ -29,7 +29,9 @@
from policyhandler.discovery import DiscoveryClient
from policyhandler.onap.audit import Audit
from policyhandler.service_activator import ServiceActivator
+from policyhandler.utils import Utils
+_LOGGER = Utils.get_logger(__file__)
def _fix_discover_config(func):
"""the decorator"""
@@ -38,7 +40,7 @@
def mocked_discover_get_value(*_):
"""monkeypatch for get from consul"""
- return copy.deepcopy(Settings.mock_config)
+ return copy.deepcopy(MockSettings.mock_config)
@wraps(func)
def wrapper(*args, **kwargs):
@@ -54,49 +56,65 @@
return func_result
return wrapper
-class Settings(object):
+class MockSettings(object):
"""init all locals"""
+ PDP_API_VERSION = "PDP_API_VERSION"
+ OLD_PDP_API_VERSION = "pdp_api_v0"
_loaded = False
- logger = None
mock_config = None
deploy_handler_instance_uuid = str(uuid.uuid4())
@staticmethod
def init():
"""init configs"""
- if Settings._loaded:
- Settings.logger.info("testing policy_handler with config: %s", Config.discovered_config)
+ if MockSettings._loaded:
+ _LOGGER.info("testing policy_handler with config: %s", Config.discovered_config)
return
- Settings._loaded = True
+ MockSettings._loaded = True
- Config.init_config()
-
- Config.consul_url = "http://unit-test-consul:850000"
+ Config.init_config("tests/etc_config.json")
with open("tests/mock_config.json", 'r') as config_json:
- Settings.mock_config = json.load(config_json)
+ MockSettings.mock_config = json.load(config_json)
- Settings.logger = logging.getLogger("policy_handler.unit_test")
- sys.stdout = LogWriter(Settings.logger.info)
- sys.stderr = LogWriter(Settings.logger.error)
+ sys.stdout = LogWriter(_LOGGER.info)
+ sys.stderr = LogWriter(_LOGGER.error)
print("print is expected to be in the log")
- Settings.logger.info("========== run_policy_handler ==========")
+ _LOGGER.info("========== run_policy_handler ==========")
Audit.init(Config.system_name, Config.LOGGER_CONFIG_FILE_PATH)
- Settings.rediscover_config()
+ MockSettings.rediscover_config()
@staticmethod
@_fix_discover_config
def rediscover_config(updated_config=None):
"""rediscover the config"""
if updated_config is not None:
- Settings.mock_config = copy.deepcopy(updated_config)
+ MockSettings.mock_config = copy.deepcopy(updated_config)
audit = Audit(req_message="rediscover_config")
Config.discover(audit)
ServiceActivator.determine_mode_of_operation(audit)
- Settings.logger.info("testing policy_handler with config: %s", Config.discovered_config)
+ _LOGGER.info("testing policy_handler with config: %s", Config.discovered_config)
audit.audit_done(" -- started")
+
+ @staticmethod
+ def setup_pdp_api(pdp_api_version=None):
+ """set the environment var for pdp_api"""
+ if Config._pdp_api_version == pdp_api_version:
+ _LOGGER.info("unchanged setup_pdp_api %s", pdp_api_version)
+ return
+
+ _LOGGER.info("setup_pdp_api %s -> %s", Config._pdp_api_version, pdp_api_version)
+
+ if pdp_api_version:
+ os.environ[MockSettings.PDP_API_VERSION] = pdp_api_version
+ elif MockSettings.PDP_API_VERSION in os.environ:
+ del os.environ[MockSettings.PDP_API_VERSION]
+ Config._pdp_api_version = pdp_api_version
+
+ importlib.reload(importlib.import_module("policyhandler.pdp_client"))
+ _LOGGER.info("done setup_pdp_api %s", Config._pdp_api_version)
diff --git a/tests/mock_tracker.py b/tests/mock_tracker.py
index 4e69afd..a5fffd5 100644
--- a/tests/mock_tracker.py
+++ b/tests/mock_tracker.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,19 +14,18 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""record all the messages going outside policy-handler during testing"""
import copy
import json
+from policyhandler.config import Config
from policyhandler.onap.audit import REQUEST_X_ECOMP_REQUESTID
-from policyhandler.policy_utils import Utils
-
-from .mock_expected import HISTORY_EXPECTED
-from .mock_settings import Settings
+from policyhandler.utils import Utils
RESPONSE = "res"
+PEP_INSTANCE = "ONAPInstance"
+_LOGGER = Utils.get_logger(__file__)
class _MockHttpRequestInResponse(object):
"""Mock http request in reponse object"""
@@ -89,20 +88,24 @@
"""record all the messages going outside policy-handler during testing"""
test_name = None
messages = []
+ test_names = []
+ validated_tests = {}
+ valid_tests = {}
@staticmethod
- def reset(test_name):
+ def reset(test_name=None):
"""remove all the messages from history"""
Tracker.test_name = test_name
Tracker.messages.clear()
+ Tracker.test_names.append(test_name)
@staticmethod
def track(message):
"""append the tracked message to the history"""
message = copy.deepcopy(message)
Tracker.messages.append(message)
- if Settings.logger:
- Settings.logger.info("tracked_message: %s", json.dumps(message, sort_keys=True))
+ if _LOGGER:
+ _LOGGER.info("tracked_message: %s", json.dumps(message, sort_keys=True))
@staticmethod
def to_string():
@@ -110,13 +113,44 @@
return json.dumps(Tracker.messages, sort_keys=True)
@staticmethod
+ def get_status(test_name=None):
+ """get the status of validation"""
+ if Tracker.valid_tests.get(test_name):
+ return "success"
+ if Tracker.validated_tests.get(test_name):
+ return "failed"
+ if test_name in Tracker.test_names:
+ return "covered"
+ return "unknown"
+
+ @staticmethod
+ def log_all_tests():
+ """log the covered and not covered test names"""
+ _LOGGER.info("-"*75)
+ _LOGGER.info("tracked test_names[%s]", len(Tracker.test_names))
+ for idx, test_name in enumerate(Tracker.test_names):
+ _LOGGER.info("%s[%s]: %s", Tracker.get_status(test_name), (idx + 1), test_name)
+
+ _LOGGER.info("not tracked test_names listed in main.mock_expected")
+ from .main.mock_expected import HISTORY_EXPECTED as main_history
+ for test_name in main_history:
+ if test_name not in Tracker.test_names:
+ _LOGGER.info("untracked: %s", test_name)
+
+ _LOGGER.info("not tracked test_names listed in pdp_api_v0.mock_expected")
+ from .pdp_api_v0.mock_expected import HISTORY_EXPECTED as pdp_api_v0_history
+ for test_name in pdp_api_v0_history:
+ if test_name not in Tracker.test_names:
+ _LOGGER.info("untracked: %s", test_name)
+
+ @staticmethod
def _hide_volatiles(obj):
"""hides the volatile field values"""
if not isinstance(obj, dict):
return obj
for key, value in obj.items():
- if key in [REQUEST_X_ECOMP_REQUESTID, RESPONSE]:
+ if key in [REQUEST_X_ECOMP_REQUESTID, RESPONSE, PEP_INSTANCE]:
obj[key] = "*"
elif isinstance(value, dict):
obj[key] = Tracker._hide_volatiles(value)
@@ -126,13 +160,21 @@
@staticmethod
def validate():
"""validate that the message history is as expected"""
- Settings.logger.info("Tracker.validate(%s)", Tracker.test_name)
+ _LOGGER.info("Tracker.validate(%s)", Tracker.test_name)
messages = [Tracker._hide_volatiles(copy.deepcopy(message))
for message in Tracker.messages]
- expected = HISTORY_EXPECTED.get(Tracker.test_name, [])
+ Tracker.validated_tests[Tracker.test_name] = True
- Settings.logger.info("messages: %s", json.dumps(messages, sort_keys=True))
- Settings.logger.info("expected: %s", json.dumps(expected, sort_keys=True))
+ if Config.is_pdp_api_default():
+ from .main.mock_expected import HISTORY_EXPECTED as main_history
+ expected = main_history.get(Tracker.test_name, [])
+ else:
+ from .pdp_api_v0.mock_expected import HISTORY_EXPECTED as pdp_api_v0_history
+ expected = pdp_api_v0_history.get(Tracker.test_name, [])
+
+ _LOGGER.info("messages: %s", json.dumps(messages, sort_keys=True))
+ _LOGGER.info("expected: %s", json.dumps(expected, sort_keys=True))
assert Utils.are_the_same(messages, expected)
- Settings.logger.info("history valid for Tracker.validate(%s)", Tracker.test_name)
+ _LOGGER.info("history valid for Tracker.validate(%s)", Tracker.test_name)
+ Tracker.valid_tests[Tracker.test_name] = True
diff --git a/tests/pdp_api_v0/__init__.py b/tests/pdp_api_v0/__init__.py
new file mode 100644
index 0000000..b8e4605
--- /dev/null
+++ b/tests/pdp_api_v0/__init__.py
@@ -0,0 +1,20 @@
+# ================================================================================
+# Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+
+# empty __init__.py so that pytest can add correct path to coverage report, -- per pytest
+# best practice guideline
diff --git a/tests/pdp_api_v0/conftest.py b/tests/pdp_api_v0/conftest.py
new file mode 100644
index 0000000..07e566f
--- /dev/null
+++ b/tests/pdp_api_v0/conftest.py
@@ -0,0 +1,133 @@
+# ============LICENSE_START=======================================================
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+"""
+startdard pytest file that contains the shared fixtures
+https://docs.pytest.org/en/latest/fixture.html
+"""
+
+import pytest
+
+from policyhandler import pdp_client
+from policyhandler.pdp_api_v0.pdp_consts import POLICY_NAME
+from policyhandler.utils import Utils
+
+from ..mock_settings import MockSettings
+from ..mock_tracker import MockHttpResponse
+from .mock_policy_engine import MockPolicyEngine2018
+from .mock_websocket import MockWebSocket
+
+_LOGGER = Utils.get_logger(__file__)
+
+@pytest.fixture(scope="session", autouse=True)
+def _auto_setup_policy_engine_pdp_api_v0():
+ """initialize the mock-policy-engine_pdp_api_v0 per the whole test session"""
+ _LOGGER.info("create _auto_setup_policy_engine_pdp_api_v0")
+ MockPolicyEngine2018.init()
+ yield _auto_setup_policy_engine_pdp_api_v0
+ _LOGGER.info("teardown _auto_setup_policy_engine_pdp_api_v0")
+
+
+@pytest.fixture(scope="module")
+def fix_pdp_api_v0():
+ """test on the old (pdp_api_v0) pdp API"""
+ _LOGGER.info("setup fix_pdp_api_v0 %s", MockSettings.OLD_PDP_API_VERSION)
+ MockSettings.setup_pdp_api(MockSettings.OLD_PDP_API_VERSION)
+
+ yield fix_pdp_api_v0
+ MockSettings.setup_pdp_api()
+ _LOGGER.info("teardown fix_pdp_api_v0 %s", MockSettings.OLD_PDP_API_VERSION)
+
+@pytest.fixture()
+def fix_pdp_post(monkeypatch):
+ """monkeyed request /getConfig to PDP"""
+ def monkeyed_policy_rest_post(uri, json=None, **kwargs):
+ """monkeypatch for the POST to policy-engine"""
+ res_json = MockPolicyEngine2018.get_config(json.get(POLICY_NAME))
+ return MockHttpResponse("post", uri, res_json, json=json, **kwargs)
+
+ _LOGGER.info("setup fix_pdp_post")
+ pdp_client.PolicyRest._lazy_init()
+ monkeypatch.setattr('policyhandler.pdp_client.PolicyRest._requests_session.post',
+ monkeyed_policy_rest_post)
+ yield fix_pdp_post
+ _LOGGER.info("teardown fix_pdp_post")
+
+@pytest.fixture()
+def fix_pdp_post_big(monkeypatch):
+ """monkeyed request /getConfig to PDP"""
+ def monkeyed_policy_rest_post(uri, **kwargs):
+ """monkeypatch for the POST to policy-engine"""
+ res_json = MockPolicyEngine2018.get_configs_all()
+ return MockHttpResponse("post", uri, res_json, **kwargs)
+
+ _LOGGER.info("setup fix_pdp_post_big")
+ pdp_client.PolicyRest._lazy_init()
+ monkeypatch.setattr('policyhandler.pdp_client.PolicyRest._requests_session.post',
+ monkeyed_policy_rest_post)
+ yield fix_pdp_post_big
+ _LOGGER.info("teardown fix_pdp_post_big")
+
+
+class MockException(Exception):
+ """mock exception"""
+ pass
+
+@pytest.fixture()
+def fix_pdp_post_boom(monkeypatch):
+ """monkeyed request /getConfig to PDP - exception"""
+ def monkeyed_policy_rest_post_boom(uri, **_):
+ """monkeypatch for the POST to policy-engine"""
+ raise MockException("fix_pdp_post_boom {}".format(uri))
+
+ _LOGGER.info("setup fix_pdp_post_boom")
+ pdp_client.PolicyRest._lazy_init()
+ monkeypatch.setattr('policyhandler.pdp_client.PolicyRest._requests_session.post',
+ monkeyed_policy_rest_post_boom)
+ yield fix_pdp_post_boom
+ _LOGGER.info("teardown fix_pdp_post_boom")
+
+
+@pytest.fixture()
+def fix_policy_receiver_websocket(monkeypatch):
+ """monkeyed websocket for policy_receiver"""
+ _LOGGER.info("setup fix_policy_receiver_websocket")
+ monkeypatch.setattr('policyhandler.pdp_api_v0.policy_listener.websocket', MockWebSocket)
+
+ yield fix_policy_receiver_websocket
+ _LOGGER.info("teardown fix_policy_receiver_websocket")
+
+class MockBoom(Exception):
+ """mock exception"""
+ pass
+
+@pytest.fixture()
+def fix_select_latest_policies_boom(monkeypatch):
+ """monkeyed exception"""
+ def monkeyed_boom(*_, **__):
+ """monkeypatch for the select_latest_policies"""
+ raise MockBoom("fix_select_latest_policies_boom")
+
+ policy_utils_path = 'policyhandler.pdp_api_v0.policy_utils.PolicyUtils'
+
+ _LOGGER.info("setup fix_select_latest_policies_boom at %s", policy_utils_path)
+
+ monkeypatch.setattr('{}.select_latest_policies'.format(policy_utils_path), monkeyed_boom)
+ monkeypatch.setattr('{}.select_latest_policy'.format(policy_utils_path), monkeyed_boom)
+ monkeypatch.setattr('{}.extract_policy_id'.format(policy_utils_path), monkeyed_boom)
+
+ yield fix_select_latest_policies_boom
+ _LOGGER.info("teardown fix_select_latest_policies_boom at %s", policy_utils_path)
diff --git a/tests/mock_expected.py b/tests/pdp_api_v0/mock_expected.py
similarity index 98%
rename from tests/mock_expected.py
rename to tests/pdp_api_v0/mock_expected.py
index c10215d..6210e10 100644
--- a/tests/mock_expected.py
+++ b/tests/pdp_api_v0/mock_expected.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,12 +14,11 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
-"""expected message history per test"""
+"""expected message history per test for pdp API 2018 and before"""
HISTORY_EXPECTED = {
- "tests.test_policy_rest::test_get_policy_latest" : [
+ "tests/pdp_api_v0/test_policy_rest.py::test_get_policy_latest" : [
{
"request": {
"headers": {
@@ -41,7 +40,7 @@
"status_code": 200
}
],
- "tests.test_policyhandler::WebServerTest::test_web_all_policies_latest": [
+ "tests/pdp_api_v0/test_policyhandler.py::WebServer2018Test::test_web_all_policies_latest": [
{
"request": {
"headers": {
@@ -78,7 +77,7 @@
"status_code": 200
}
],
- "tests.test_policyhandler::WebServerTest::test_web_policies_latest": [
+ "tests/pdp_api_v0/test_policyhandler.py::WebServer2018Test::test_web_policies_latest": [
{
"request": {
"headers": {
@@ -100,7 +99,7 @@
"status_code": 200
}
],
- "tests.test_policyhandler::WebServerTest::test_web_policy_latest": [
+ "tests/pdp_api_v0/test_policyhandler.py::WebServer2018Test::test_web_policy_latest": [
{
"request": {
"headers": {
@@ -122,7 +121,7 @@
"status_code": 200
}
],
- "tests.test_policyhandler::WebServerTest::test_zzz_get_catch_up": [
+ "tests/pdp_api_v0/test_policyhandler.py::WebServer2018Test::test_zzz_get_catch_up": [
{
"request": {
"headers": {
@@ -560,7 +559,7 @@
"status_code": 200
}
],
- "tests.test_policyhandler::WebServerTest::test_zzz_policy_updates_and_catch_ups": [
+ "tests/pdp_api_v0/test_policyhandler.py::WebServer2018Test::test_zzz_policy_updates_and_catch_ups": [
{
"request": {
"headers": {
@@ -946,7 +945,7 @@
"status_code": 200
}
],
- "tests.test_policyhandler::WebServerTest::test_zzzzz_shutdown": [
+ "tests/pdp_api_v0/test_policyhandler.py::WebServer2018Test::test_zzzzz_shutdown": [
{
"request": {
"headers": {
@@ -1332,7 +1331,7 @@
"status_code": 200
}
],
- "tests.test_policyhandler::WebServerTest::test_zzz_catch_up_on_deploy_handler_changed": [
+ "tests/pdp_api_v0/test_policyhandler.py::WebServer2018Test::test_zzz_catch_up_on_deploy_handler_changed": [
{
"request": {
"headers": {
@@ -1973,7 +1972,7 @@
"status_code": 200
}
],
- "tests.test_pz_catch_up::test_catch_up_failed_dh": [
+ "tests/pdp_api_v0/test_pz_catch_up.py::test_catch_up_failed_dh": [
{
"request": {
"headers": {
@@ -2524,7 +2523,7 @@
"status_code": 413
}
],
- "tests.test_pz_catch_up::test_catch_up_dh_404": [
+ "tests/pdp_api_v0/test_pz_catch_up.py::test_catch_up_dh_404": [
{
"request": {
"headers": {
@@ -2571,7 +2570,7 @@
"status_code": 200
}
],
- "tests.test_pz_pdp_boom::WebServerPDPBoomTest::test_web_all_policies_latest": [
+ "tests/pdp_api_v0/test_pz_pdp_boom.py::WebServerPDPBoom2018Test::test_web_all_policies_latest": [
{
"request": {
"headers": {
@@ -2588,7 +2587,7 @@
"status_code": 200
}
],
- "tests.test_pz_pdp_boom::WebServerPDPBoomTest::test_zzz_catch_up_on_deploy_handler_changed": [
+ "tests/pdp_api_v0/test_pz_pdp_boom.py::WebServerPDPBoom2018Test::test_zzz_catch_up_on_deploy_handler_changed": [
{
"request": {
"headers": {
@@ -2650,7 +2649,7 @@
"status_code": 200
}
],
- "tests.test_pz_pdp_boom::WebServerPDPBoomTest::test_zzz_get_catch_up": [
+ "tests/pdp_api_v0/test_pz_pdp_boom.py::WebServerPDPBoom2018Test::test_zzz_get_catch_up": [
{
"request": {
"headers": {
@@ -2682,7 +2681,7 @@
"status_code": 200
}
],
- "tests.test_pz_pdp_boom::WebServerPDPBoomTest::test_zzz_policy_updates_and_catch_ups": [
+ "tests/pdp_api_v0/test_pz_pdp_boom.py::WebServerPDPBoom2018Test::test_zzz_policy_updates_and_catch_ups": [
{
"request": {
"headers": {
@@ -2714,7 +2713,7 @@
"status_code": 200
}
],
- "tests.test_pz_pdp_boom::WebServerPDPBoomTest::test_zzzzz_shutdown": [
+ "tests/pdp_api_v0/test_pz_pdp_boom.py::WebServerPDPBoom2018Test::test_zzzzz_shutdown": [
{
"request": {
"headers": {
@@ -2746,7 +2745,7 @@
"status_code": 200
}
],
- "tests.test_pz_ph_boom::WebServerInternalBoomTest::test_web_all_policies_latest": [
+ "tests/pdp_api_v0/test_pz_ph_boom.py::WebServerInternalBoom2018Test::test_web_all_policies_latest": [
{
"request": {
"headers": {
@@ -2783,7 +2782,7 @@
"status_code": 200
}
],
- "tests.test_pz_ph_boom::WebServerInternalBoomTest::test_web_policies_latest": [
+ "tests/pdp_api_v0/test_pz_ph_boom.py::WebServerInternalBoom2018Test::test_web_policies_latest": [
{
"request": {
"headers": {
@@ -2805,7 +2804,7 @@
"status_code": 200
}
],
- "tests.test_pz_ph_boom::WebServerInternalBoomTest::test_web_policy_latest": [
+ "tests/pdp_api_v0/test_pz_ph_boom.py::WebServerInternalBoom2018Test::test_web_policy_latest": [
{
"request": {
"headers": {
@@ -2827,7 +2826,7 @@
"status_code": 200
}
],
- "tests.test_pz_ph_boom::WebServerInternalBoomTest::test_zzz_catch_up_on_deploy_handler_changed": [
+ "tests/pdp_api_v0/test_pz_ph_boom.py::WebServerInternalBoom2018Test::test_zzz_catch_up_on_deploy_handler_changed": [
{
"request": {
"headers": {
@@ -2864,7 +2863,7 @@
"status_code": 200
}
],
- "tests.test_pz_ph_boom::WebServerInternalBoomTest::test_zzz_get_catch_up": [
+ "tests/pdp_api_v0/test_pz_ph_boom.py::WebServerInternalBoom2018Test::test_zzz_get_catch_up": [
{
"request": {
"headers": {
@@ -2936,7 +2935,7 @@
"status_code": 200
}
],
- "tests.test_pz_ph_boom::WebServerInternalBoomTest::test_zzz_policy_updates_and_catch_ups": [
+ "tests/pdp_api_v0/test_pz_ph_boom.py::WebServerInternalBoom2018Test::test_zzz_policy_updates_and_catch_ups": [
{
"request": {
"headers": {
@@ -2973,7 +2972,7 @@
"status_code": 200
}
],
- "tests.test_pz_ph_boom::WebServerInternalBoomTest::test_zzzzz_shutdown": [
+ "tests/pdp_api_v0/test_pz_ph_boom.py::WebServerInternalBoom2018Test::test_zzzzz_shutdown": [
{
"request": {
"headers": {
diff --git a/tests/mock_policy_engine.py b/tests/pdp_api_v0/mock_policy_engine.py
similarity index 61%
rename from tests/mock_policy_engine.py
rename to tests/pdp_api_v0/mock_policy_engine.py
index d57d613..cdded14 100644
--- a/tests/mock_policy_engine.py
+++ b/tests/pdp_api_v0/mock_policy_engine.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,24 +14,97 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""mocking for the policy-engine - shared by many tests"""
import copy
import json
import re
-from policyhandler.policy_consts import (POLICY_BODY, POLICY_CONFIG, POLICY_ID,
- POLICY_NAME, POLICY_VERSION)
-from policyhandler.policy_utils import PolicyUtils
+from policyhandler.pdp_api_v0.pdp_consts import (POLICY_CONFIG, POLICY_NAME,
+ POLICY_VERSION)
+from policyhandler.pdp_api_v0.policy_utils import PolicyUtils
+from policyhandler.policy_consts import POLICY_BODY, POLICY_ID
+from policyhandler.utils import Utils
-from .mock_settings import Settings
+_LOGGER = Utils.get_logger(__file__)
-class MonkeyPolicyBody(object):
- """policy body that policy-engine returns"""
+class MockPolicyEngine2018(object):
+ """pretend this is the policy-engine"""
+ scope_prefix = "test_scope_prefix.Config_"
+ LOREM_IPSUM = """Lorem ipsum dolor sit amet consectetur ametist""".split()
+ LONG_TEXT = "0123456789" * 100
+ _policies = []
+
+ _inited = False
+
@staticmethod
- def create_policy_body(policy_id, policy_version=1):
+ def init():
+ """init collection of policies: policy_version = policy_index + 1"""
+ if MockPolicyEngine2018._inited:
+ return
+ MockPolicyEngine2018._inited = True
+
+ MockPolicyEngine2018._policies = [
+ MockPolicyEngine2018._create_policy_body(
+ MockPolicyEngine2018.scope_prefix + policy_id, policy_index + 1)
+ for policy_id in MockPolicyEngine2018.LOREM_IPSUM
+ for policy_index in range(1 + MockPolicyEngine2018.LOREM_IPSUM.index(policy_id))]
+ _LOGGER.info("_policies: %s", json.dumps(MockPolicyEngine2018._policies))
+
+ @staticmethod
+ def get_config(policy_name):
+ """find policy the way the policy-engine finds"""
+ if not policy_name:
+ return []
+ return [copy.deepcopy(policy)
+ for policy in MockPolicyEngine2018._policies
+ if re.match(policy_name, policy[POLICY_NAME])]
+
+ @staticmethod
+ def get_configs_all():
+ """get all policies the way the policy-engine finds"""
+ policies = [copy.deepcopy(policy)
+ for policy in MockPolicyEngine2018._policies]
+ for policy in policies:
+ policy["config"] = MockPolicyEngine2018.LONG_TEXT
+ return policies
+
+ @staticmethod
+ def get_policy_id(policy_index):
+ """get the policy_id by policy_index"""
+ return (MockPolicyEngine2018.scope_prefix
+ + MockPolicyEngine2018.LOREM_IPSUM[
+ policy_index % len(MockPolicyEngine2018.LOREM_IPSUM)])
+
+ @staticmethod
+ def gen_policy_latest(policy_index, version_offset=0):
+ """generate the policy response from policy-handler by policy_index = version - 1"""
+ policy_id = MockPolicyEngine2018.get_policy_id(policy_index)
+ policy = {
+ POLICY_ID: policy_id,
+ POLICY_BODY: MockPolicyEngine2018._create_policy_body(
+ policy_id, policy_index + 1 - version_offset)
+ }
+ return policy_id, PolicyUtils.parse_policy_config(policy)
+
+ @staticmethod
+ def gen_all_policies_latest(version_offset=0):
+ """generate all latest policies"""
+ return dict(
+ MockPolicyEngine2018.gen_policy_latest(policy_index, version_offset=version_offset)
+ for policy_index in range(len(MockPolicyEngine2018.LOREM_IPSUM))
+ )
+
+ @staticmethod
+ def gen_policies_latest(match_to_policy_name):
+ """generate all latest policies"""
+ return dict((k, v)
+ for k, v in MockPolicyEngine2018.gen_all_policies_latest().items()
+ if re.match(match_to_policy_name, k))
+
+ @staticmethod
+ def _create_policy_body(policy_id, policy_version=1):
"""returns a fake policy-body"""
prev_ver = str(policy_version - 1)
this_ver = str(policy_version)
@@ -55,77 +128,3 @@
"responseAttributes": {},
"property": None
}
-
-
-class MockPolicyEngine(object):
- """pretend this is the policy-engine"""
- scope_prefix = "test_scope_prefix.Config_"
- LOREM_IPSUM = """Lorem ipsum dolor sit amet consectetur ametist""".split()
- LONG_TEXT = "0123456789" * 100
- _policies = []
-
- _inited = False
-
- @staticmethod
- def init():
- """init static vars"""
- if MockPolicyEngine._inited:
- return
- MockPolicyEngine._inited = True
-
- MockPolicyEngine._policies = [
- MonkeyPolicyBody.create_policy_body(
- MockPolicyEngine.scope_prefix + policy_id, policy_index + 1)
- for policy_id in MockPolicyEngine.LOREM_IPSUM
- for policy_index in range(1 + MockPolicyEngine.LOREM_IPSUM.index(policy_id))]
- Settings.logger.info("MockPolicyEngine._policies: %s",
- json.dumps(MockPolicyEngine._policies))
-
- @staticmethod
- def get_config(policy_name):
- """find policy the way the policy-engine finds"""
- if not policy_name:
- return []
- return [copy.deepcopy(policy)
- for policy in MockPolicyEngine._policies
- if re.match(policy_name, policy[POLICY_NAME])]
-
- @staticmethod
- def get_configs_all():
- """get all policies the way the policy-engine finds"""
- policies = [copy.deepcopy(policy)
- for policy in MockPolicyEngine._policies]
- for policy in policies:
- policy["config"] = MockPolicyEngine.LONG_TEXT
- return policies
-
- @staticmethod
- def get_policy_id(policy_index):
- """get the policy_id by index"""
- return (MockPolicyEngine.scope_prefix
- + MockPolicyEngine.LOREM_IPSUM[
- policy_index % len(MockPolicyEngine.LOREM_IPSUM)])
-
- @staticmethod
- def gen_policy_latest(policy_index, version_offset=0):
- """generate the policy response by policy_index = version - 1"""
- policy_id = MockPolicyEngine.get_policy_id(policy_index)
- policy = {
- POLICY_ID: policy_id,
- POLICY_BODY: MonkeyPolicyBody.create_policy_body(
- policy_id, policy_index + 1 - version_offset)
- }
- return policy_id, PolicyUtils.parse_policy_config(policy)
-
- @staticmethod
- def gen_all_policies_latest(version_offset=0):
- """generate all latest policies"""
- return dict(MockPolicyEngine.gen_policy_latest(policy_index, version_offset=version_offset)
- for policy_index in range(len(MockPolicyEngine.LOREM_IPSUM)))
-
- @staticmethod
- def gen_policies_latest(match_to_policy_name):
- """generate all latest policies"""
- return dict((k, v)
- for k, v in MockPolicyEngine.gen_all_policies_latest().items()
- if re.match(match_to_policy_name, k))
diff --git a/tests/mock_websocket.py b/tests/pdp_api_v0/mock_websocket.py
similarity index 70%
rename from tests/mock_websocket.py
rename to tests/pdp_api_v0/mock_websocket.py
index ac64ed8..17f3bbe 100644
--- a/tests/mock_websocket.py
+++ b/tests/pdp_api_v0/mock_websocket.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,19 +14,20 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""mocking for the websocket - for testing of policy-engine"""
import json
import time
-from policyhandler.policy_consts import POLICY_NAME
-from policyhandler.policy_receiver import (LOADED_POLICIES, POLICY_VER,
- REMOVED_POLICIES)
+from policyhandler.pdp_api_v0.pdp_consts import POLICY_NAME
+from policyhandler.pdp_api_v0.policy_listener import (LOADED_POLICIES,
+ POLICY_VER,
+ REMOVED_POLICIES)
+from policyhandler.utils import Utils
-from .mock_policy_engine import MockPolicyEngine
-from .mock_settings import Settings
+from .mock_policy_engine import MockPolicyEngine2018
+_LOGGER = Utils.get_logger(__file__)
class MockWebSocket(object):
"""Mock websocket"""
@@ -40,14 +41,14 @@
message = {
LOADED_POLICIES: [
{POLICY_NAME: "{0}.{1}.xml".format(
- MockPolicyEngine.get_policy_id(policy_index), policy_index + 1),
+ MockPolicyEngine2018.get_policy_id(policy_index), policy_index + 1),
POLICY_VER: str(policy_index + 1)}
for policy_index in updated_indexes or []
],
REMOVED_POLICIES : []
}
message = json.dumps(message)
- Settings.logger.info("send_notification: %s", message)
+ _LOGGER.info("send_notification: %s", message)
MockWebSocket.on_message(None, message)
@staticmethod
@@ -71,18 +72,18 @@
self.on_error = on_error
self.on_pong = on_pong
self.sock = MockWebSocket.MockSocket()
- Settings.logger.info("MockWebSocket for: %s", self.web_socket_url)
+ _LOGGER.info("MockWebSocket for: %s", self.web_socket_url)
- def run_forever(self, sslopt=None):
+ def run_forever(self, sslopt=None, ping_interval=None):
"""forever in the loop"""
- Settings.logger.info("MockWebSocket run_forever with sslopt=%s...",
- json.dumps(sslopt))
+ _LOGGER.info("MockWebSocket run_forever with sslopt=%s, ping_interval=%s...",
+ json.dumps(sslopt), ping_interval)
counter = 0
while self.sock.connected:
counter += 1
- Settings.logger.info("MockWebSocket sleep %s...", counter)
+ _LOGGER.info("MockWebSocket sleep %s...", counter)
time.sleep(5)
- Settings.logger.info("MockWebSocket exit %s", counter)
+ _LOGGER.info("MockWebSocket exit %s", counter)
def close(self):
"""close socket"""
diff --git a/tests/test_policy_rest.py b/tests/pdp_api_v0/test_policy_rest.py
similarity index 64%
copy from tests/test_policy_rest.py
copy to tests/pdp_api_v0/test_policy_rest.py
index ac2529a..67b06ae 100644
--- a/tests/test_policy_rest.py
+++ b/tests/pdp_api_v0/test_policy_rest.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,34 +14,34 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test policy_rest methods directly"""
import json
import pytest
+from policyhandler import pdp_client
from policyhandler.onap.audit import Audit
-from policyhandler.policy_rest import PolicyRest
-from policyhandler.policy_utils import Utils
+from policyhandler.utils import Utils
-from .mock_policy_engine import MockPolicyEngine
-from .mock_settings import Settings
-from .mock_tracker import Tracker
+from ..mock_tracker import Tracker
+from .mock_policy_engine import MockPolicyEngine2018
+_LOGGER = Utils.get_logger(__file__)
-@pytest.mark.usefixtures("fix_pdp_post")
+@pytest.mark.usefixtures("fix_pdp_api_v0", "fix_pdp_post")
def test_get_policy_latest():
"""test /policy_latest/<policy-id>"""
- policy_id, expected_policy = MockPolicyEngine.gen_policy_latest(3)
+ policy_id, expected_policy = MockPolicyEngine2018.gen_policy_latest(3)
audit = Audit(job_name="test_get_policy_latest",
req_message="get /policy_latest/{}".format(policy_id or ""))
- policy_latest = PolicyRest.get_latest_policy((audit, policy_id, None, None)) or {}
+
+ policy_latest = pdp_client.PolicyRest.get_latest_policy((audit, policy_id, None, None)) or {}
audit.audit_done(result=json.dumps(policy_latest))
- Settings.logger.info("expected_policy: %s", json.dumps(expected_policy))
- Settings.logger.info("policy_latest: %s", json.dumps(policy_latest))
+ _LOGGER.info("expected_policy: %s", json.dumps(expected_policy))
+ _LOGGER.info("policy_latest: %s", json.dumps(policy_latest))
assert Utils.are_the_same(policy_latest, expected_policy)
Tracker.validate()
diff --git a/tests/test_policyhandler.py b/tests/pdp_api_v0/test_policyhandler.py
similarity index 62%
rename from tests/test_policyhandler.py
rename to tests/pdp_api_v0/test_policyhandler.py
index 4743a6e..2b2629b 100644
--- a/tests/test_policyhandler.py
+++ b/tests/pdp_api_v0/test_policyhandler.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2017-2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2017-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test of the package for policy-handler of DCAE-Controller"""
@@ -22,24 +21,31 @@
import time
import uuid
-import pytest
import cherrypy
+import pytest
from cherrypy.test.helper import CPWebCase
+from policyhandler.config import Config
from policyhandler.onap.audit import REQUEST_X_ECOMP_REQUESTID, Audit
-from policyhandler.policy_consts import LATEST_POLICIES, POLICY_NAME
+from policyhandler.pdp_api_v0.pdp_consts import POLICY_NAME
+from policyhandler.policy_consts import LATEST_POLICIES
from policyhandler.policy_receiver import PolicyReceiver
-from policyhandler.policy_utils import Utils
+from policyhandler.utils import Utils
from policyhandler.web_server import _PolicyWeb
-from .mock_policy_engine import MockPolicyEngine
-from .mock_settings import Settings
-from .mock_tracker import Tracker
+from ..mock_settings import MockSettings
+from ..mock_tracker import Tracker
+from .mock_policy_engine import MockPolicyEngine2018
from .mock_websocket import MockWebSocket
+_LOGGER = Utils.get_logger(__file__)
-@pytest.mark.usefixtures("fix_pdp_post", "fix_discovery")
-class WebServerTest(CPWebCase):
+@pytest.mark.usefixtures(
+ "fix_pdp_api_v0",
+ "fix_pdp_post",
+ "fix_discovery"
+)
+class WebServer2018Test(CPWebCase):
"""testing the web-server - runs tests in alphabetical order of method names"""
def setup_server():
"""setup the web-server"""
@@ -50,57 +56,62 @@
def test_web_healthcheck(self):
"""test /healthcheck"""
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
- Settings.logger.info("got healthcheck: %s", self.body)
+ _LOGGER.info("healthcheck result: %s", result)
+ _LOGGER.info("got healthcheck: %s", self.body)
self.assertStatus('200 OK')
Tracker.validate()
def test_web_policy_latest(self):
"""test /policy_latest/<policy-id>"""
- policy_id, expected_policy = MockPolicyEngine.gen_policy_latest(3)
+ policy_id, expected_policy = MockPolicyEngine2018.gen_policy_latest(3)
self.getPage("/policy_latest/{0}".format(policy_id or ""))
self.assertStatus('200 OK')
policy_latest = json.loads(self.body)
- Settings.logger.info("policy_latest: %s", self.body)
- Settings.logger.info("expected_policy: %s", json.dumps(expected_policy))
+ _LOGGER.info("policy_latest: %s", self.body)
+ _LOGGER.info("expected_policy: %s", json.dumps(expected_policy))
assert Utils.are_the_same(policy_latest, expected_policy)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
@pytest.mark.usefixtures("fix_deploy_handler")
def test_web_all_policies_latest(self):
"""test GET /policies_latest"""
- expected_policies = MockPolicyEngine.gen_all_policies_latest()
+ expected_policies = MockPolicyEngine2018.gen_all_policies_latest()
result = self.getPage("/policies_latest")
- Settings.logger.info("result: %s", result)
- Settings.logger.info("body: %s", self.body)
+ _LOGGER.info("result: %s", result)
+ _LOGGER.info("body: %s", self.body)
+
+ if Config.is_pdp_api_default():
+ self.assertStatus('404 Not Found')
+ return
+
self.assertStatus('200 OK')
policies_latest = json.loads(self.body)
self.assertIn(LATEST_POLICIES, policies_latest)
policies_latest = policies_latest[LATEST_POLICIES]
- Settings.logger.info("policies_latest: %s", json.dumps(policies_latest))
- Settings.logger.info("expected_policies: %s", json.dumps(expected_policies))
+ _LOGGER.info("policies_latest: %s", json.dumps(policies_latest))
+ _LOGGER.info("expected_policies: %s", json.dumps(expected_policies))
assert Utils.are_the_same(policies_latest, expected_policies)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
def test_web_policies_latest(self):
"""test POST /policies_latest with policyName"""
- match_to_policy_name = MockPolicyEngine.scope_prefix + "amet.*"
- expected_policies = MockPolicyEngine.gen_policies_latest(match_to_policy_name)
+ match_to_policy_name = MockPolicyEngine2018.scope_prefix + "amet.*"
+ expected_policies = MockPolicyEngine2018.gen_policies_latest(match_to_policy_name)
body = json.dumps({POLICY_NAME: match_to_policy_name})
result = self.getPage("/policies_latest", method='POST',
@@ -110,43 +121,52 @@
("Content-Type", "application/json"),
('Content-Length', str(len(body)))
])
- Settings.logger.info("result: %s", result)
- Settings.logger.info("body: %s", self.body)
+ _LOGGER.info("result: %s", result)
+ _LOGGER.info("body: %s", self.body)
+
+ if Config.is_pdp_api_default():
+ self.assertStatus('404 Not Found')
+ return
+
self.assertStatus('200 OK')
policies_latest = json.loads(self.body)[LATEST_POLICIES]
- Settings.logger.info("policies_latest: %s", json.dumps(policies_latest))
- Settings.logger.info("expected_policies: %s", json.dumps(expected_policies))
+ _LOGGER.info("policies_latest: %s", json.dumps(policies_latest))
+ _LOGGER.info("expected_policies: %s", json.dumps(expected_policies))
assert Utils.are_the_same(policies_latest, expected_policies)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_policy_updates_and_catch_ups(self):
"""test run policy handler with policy updates and catchups"""
- Settings.logger.info("start policy_updates_and_catch_ups")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start policy_updates_and_catch_ups")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_policy_updates_and_catch_ups",
req_message="start policy_updates_and_catch_ups")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1, 3, 5])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.logger.info("sleep 10 before shutdown...")
+ _LOGGER.info("sleep 10 before shutdown...")
time.sleep(10)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -157,31 +177,35 @@
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_catch_up_on_deploy_handler_changed(self):
"""test run policy handler with deployment-handler changed underneath"""
- Settings.logger.info("start zzz_catch_up_on_deploy_handler_changed")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start zzz_catch_up_on_deploy_handler_changed")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_catch_up_on_deploy_handler_changed",
req_message="start zzz_catch_up_on_deploy_handler_changed")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.deploy_handler_instance_uuid = str(uuid.uuid4())
- Settings.logger.info("new deploy-handler uuid=%s", Settings.deploy_handler_instance_uuid)
+ MockSettings.deploy_handler_instance_uuid = str(uuid.uuid4())
+ _LOGGER.info("new deploy-handler uuid=%s", MockSettings.deploy_handler_instance_uuid)
MockWebSocket.send_notification([2, 4])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.logger.info("sleep 5 before shutdown...")
+ _LOGGER.info("sleep 5 before shutdown...")
time.sleep(5)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -192,21 +216,25 @@
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_get_catch_up(self):
"""test /catch_up"""
- Settings.logger.info("start /catch_up")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start /catch_up")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_get_catch_up", req_message="start /catch_up")
PolicyReceiver.run(audit)
time.sleep(5)
result = self.getPage("/catch_up")
- Settings.logger.info("catch_up result: %s", result)
+ _LOGGER.info("catch_up result: %s", result)
self.assertStatus('200 OK')
- Settings.logger.info("got catch_up: %s", self.body)
+ _LOGGER.info("got catch_up: %s", self.body)
- Settings.logger.info("sleep 5 before shutdown...")
+ _LOGGER.info("sleep 5 before shutdown...")
time.sleep(5)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -220,29 +248,33 @@
"fix_cherrypy_engine_exit")
def test_zzzzz_shutdown(self):
"""test shutdown"""
- Settings.logger.info("start shutdown")
+ _LOGGER.info("start shutdown")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzzzz_shutdown", req_message="start shutdown")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1, 3, 5])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
- WebServerTest.do_gc_test = False
- Settings.logger.info("shutdown...")
+ WebServer2018Test.do_gc_test = False
+ _LOGGER.info("shutdown...")
audit.audit_done("shutdown")
result = self.getPage("/shutdown")
- Settings.logger.info("shutdown result: %s", result)
+ _LOGGER.info("shutdown result: %s", result)
self.assertStatus('200 OK')
- Settings.logger.info("got shutdown: %s", self.body)
+ _LOGGER.info("got shutdown: %s", self.body)
time.sleep(1)
assert not PolicyReceiver.is_running()
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
Tracker.validate()
diff --git a/tests/test_pz_catch_up.py b/tests/pdp_api_v0/test_pz_catch_up.py
similarity index 73%
rename from tests/test_pz_catch_up.py
rename to tests/pdp_api_v0/test_pz_catch_up.py
index 89be9bb..3b37af5 100644
--- a/tests/test_pz_catch_up.py
+++ b/tests/pdp_api_v0/test_pz_catch_up.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test policy catch_up methods directly"""
import json
@@ -22,14 +21,17 @@
import pytest
+from policyhandler.config import Config
from policyhandler.onap.audit import Audit
from policyhandler.policy_receiver import PolicyReceiver
+from policyhandler.utils import Utils
-from .mock_settings import Settings
-from .mock_tracker import Tracker
+from ..mock_tracker import Tracker
+_LOGGER = Utils.get_logger(__file__)
@pytest.mark.usefixtures(
+ "fix_pdp_api_v0",
"fix_auto_catch_up",
"fix_discovery",
"fix_pdp_post_big",
@@ -38,19 +40,23 @@
)
def test_catch_up_failed_dh():
"""test run policy handler with catchups and failed deployment-handler"""
- Settings.logger.info("start test_catch_up_failed_dh")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start test_catch_up_failed_dh")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_catch_up_failed_dh",
req_message="start test_catch_up_failed_dh")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep 12 before shutdown...")
+ _LOGGER.info("sleep 12 before shutdown...")
time.sleep(12)
health = audit.health(full=True)
audit.audit_done(result=json.dumps(health))
- Settings.logger.info("healthcheck: %s", json.dumps(health))
+ _LOGGER.info("healthcheck: %s", json.dumps(health))
assert bool(health)
PolicyReceiver.shutdown(audit)
@@ -58,11 +64,12 @@
assert not PolicyReceiver.is_running()
health = audit.health(full=True)
- Settings.logger.info("healthcheck: %s", json.dumps(health))
+ _LOGGER.info("healthcheck: %s", json.dumps(health))
Tracker.validate()
@pytest.mark.usefixtures(
+ "fix_pdp_api_v0",
"fix_auto_catch_up",
"fix_discovery",
"fix_pdp_post",
@@ -71,19 +78,23 @@
)
def test_catch_up_dh_404():
"""test run policy handler with catchups and failed deployment-handler"""
- Settings.logger.info("start test_catch_up_dh_404")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start test_catch_up_dh_404")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_catch_up_dh_404",
req_message="start test_catch_up_dh_404")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep 12 before shutdown...")
+ _LOGGER.info("sleep 12 before shutdown...")
time.sleep(12)
health = audit.health(full=True)
audit.audit_done(result=json.dumps(health))
- Settings.logger.info("healthcheck: %s", json.dumps(health))
+ _LOGGER.info("healthcheck: %s", json.dumps(health))
assert bool(health)
PolicyReceiver.shutdown(audit)
@@ -91,6 +102,6 @@
assert not PolicyReceiver.is_running()
health = audit.health(full=True)
- Settings.logger.info("healthcheck: %s", json.dumps(health))
+ _LOGGER.info("healthcheck: %s", json.dumps(health))
Tracker.validate()
diff --git a/tests/test_pz_pdp_boom.py b/tests/pdp_api_v0/test_pz_pdp_boom.py
similarity index 65%
rename from tests/test_pz_pdp_boom.py
rename to tests/pdp_api_v0/test_pz_pdp_boom.py
index 1b6f150..effadc2 100644
--- a/tests/test_pz_pdp_boom.py
+++ b/tests/pdp_api_v0/test_pz_pdp_boom.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,34 +14,37 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test policyhandler web-server when pdp booms = fails"""
import json
import time
import uuid
-import pytest
import cherrypy
+import pytest
from cherrypy.test.helper import CPWebCase
+from policyhandler.config import Config
from policyhandler.onap.audit import (REQUEST_X_ECOMP_REQUESTID, Audit,
AuditHttpCode)
-from policyhandler.policy_consts import POLICY_NAME
+from policyhandler.pdp_api_v0.pdp_consts import POLICY_NAME
from policyhandler.policy_receiver import PolicyReceiver
+from policyhandler.utils import Utils
from policyhandler.web_server import _PolicyWeb
-from .mock_policy_engine import MockPolicyEngine
-from .mock_settings import Settings
-from .mock_tracker import Tracker
+from ..mock_settings import MockSettings
+from ..mock_tracker import Tracker
+from .mock_policy_engine import MockPolicyEngine2018
from .mock_websocket import MockWebSocket
+_LOGGER = Utils.get_logger(__file__)
@pytest.mark.usefixtures(
+ "fix_pdp_api_v0",
"fix_discovery",
"fix_pdp_post_boom"
)
-class WebServerPDPBoomTest(CPWebCase):
+class WebServerPDPBoom2018Test(CPWebCase):
"""testing the web-server - runs tests in alphabetical order of method names"""
def setup_server():
"""setup the web-server"""
@@ -52,21 +55,21 @@
def test_web_healthcheck(self):
"""test /healthcheck"""
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
- Settings.logger.info("got healthcheck: %s", self.body)
+ _LOGGER.info("healthcheck result: %s", result)
+ _LOGGER.info("got healthcheck: %s", self.body)
self.assertStatus('200 OK')
Tracker.validate()
def test_web_policy_latest(self):
"""test /policy_latest/<policy-id>"""
- policy_id, _ = MockPolicyEngine.gen_policy_latest(3)
+ policy_id, _ = MockPolicyEngine2018.gen_policy_latest(3)
self.getPage("/policy_latest/{0}".format(policy_id or ""))
self.assertStatus(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
@@ -74,18 +77,23 @@
def test_web_all_policies_latest(self):
"""test GET /policies_latest"""
result = self.getPage("/policies_latest")
- Settings.logger.info("result: %s", result)
- Settings.logger.info("body: %s", self.body)
+ _LOGGER.info("result: %s", result)
+ _LOGGER.info("body: %s", self.body)
+
+ if Config.is_pdp_api_default():
+ self.assertStatus('404 Not Found')
+ return
+
self.assertStatus(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
def test_web_policies_latest(self):
"""test POST /policies_latest with policyName"""
- match_to_policy_name = MockPolicyEngine.scope_prefix + "amet.*"
+ match_to_policy_name = MockPolicyEngine2018.scope_prefix + "amet.*"
body = json.dumps({POLICY_NAME: match_to_policy_name})
result = self.getPage("/policies_latest", method='POST',
@@ -95,36 +103,45 @@
("Content-Type", "application/json"),
('Content-Length', str(len(body)))
])
- Settings.logger.info("result: %s", result)
- Settings.logger.info("body: %s", self.body)
+ _LOGGER.info("result: %s", result)
+ _LOGGER.info("body: %s", self.body)
+
+ if Config.is_pdp_api_default():
+ self.assertStatus('404 Not Found')
+ return
+
self.assertStatus(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_policy_updates_and_catch_ups(self):
"""test run policy handler with policy updates and catchups"""
- Settings.logger.info("start policy_updates_and_catch_ups")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start policy_updates_and_catch_ups")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_policy_updates_and_catch_ups",
req_message="start policy_updates_and_catch_ups")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1, 3, 5])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.logger.info("sleep 10 before shutdown...")
+ _LOGGER.info("sleep 10 before shutdown...")
time.sleep(10)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -135,31 +152,35 @@
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_catch_up_on_deploy_handler_changed(self):
"""test run policy handler with deployment-handler changed underneath"""
- Settings.logger.info("start zzz_catch_up_on_deploy_handler_changed")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start zzz_catch_up_on_deploy_handler_changed")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_catch_up_on_deploy_handler_changed",
req_message="start zzz_catch_up_on_deploy_handler_changed")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.deploy_handler_instance_uuid = str(uuid.uuid4())
- Settings.logger.info("new deploy-handler uuid=%s", Settings.deploy_handler_instance_uuid)
+ MockSettings.deploy_handler_instance_uuid = str(uuid.uuid4())
+ _LOGGER.info("new deploy-handler uuid=%s", MockSettings.deploy_handler_instance_uuid)
MockWebSocket.send_notification([2, 4])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.logger.info("sleep 5 before shutdown...")
+ _LOGGER.info("sleep 5 before shutdown...")
time.sleep(5)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -170,21 +191,25 @@
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_get_catch_up(self):
"""test /catch_up"""
- Settings.logger.info("start /catch_up")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start /catch_up")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_get_catch_up", req_message="start /catch_up")
PolicyReceiver.run(audit)
time.sleep(5)
result = self.getPage("/catch_up")
- Settings.logger.info("catch_up result: %s", result)
+ _LOGGER.info("catch_up result: %s", result)
self.assertStatus('200 OK')
- Settings.logger.info("got catch_up: %s", self.body)
+ _LOGGER.info("got catch_up: %s", self.body)
- Settings.logger.info("sleep 5 before shutdown...")
+ _LOGGER.info("sleep 5 before shutdown...")
time.sleep(5)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -198,29 +223,33 @@
"fix_cherrypy_engine_exit")
def test_zzzzz_shutdown(self):
"""test shutdown"""
- Settings.logger.info("start shutdown")
+ _LOGGER.info("start shutdown")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzzzz_shutdown", req_message="start shutdown")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1, 3, 5])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
- WebServerPDPBoomTest.do_gc_test = False
- Settings.logger.info("shutdown...")
+ WebServerPDPBoom2018Test.do_gc_test = False
+ _LOGGER.info("shutdown...")
audit.audit_done("shutdown")
result = self.getPage("/shutdown")
- Settings.logger.info("shutdown result: %s", result)
+ _LOGGER.info("shutdown result: %s", result)
self.assertStatus('200 OK')
- Settings.logger.info("got shutdown: %s", self.body)
+ _LOGGER.info("got shutdown: %s", self.body)
time.sleep(1)
assert not PolicyReceiver.is_running()
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
Tracker.validate()
diff --git a/tests/test_pz_ph_boom.py b/tests/pdp_api_v0/test_pz_ph_boom.py
similarity index 65%
rename from tests/test_pz_ph_boom.py
rename to tests/pdp_api_v0/test_pz_ph_boom.py
index c392b3a..4203110 100644
--- a/tests/test_pz_ph_boom.py
+++ b/tests/pdp_api_v0/test_pz_ph_boom.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,36 +14,38 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test policyhandler web-server when the policy-handler booms = fails"""
import json
import time
import uuid
-
-import pytest
import cherrypy
+import pytest
from cherrypy.test.helper import CPWebCase
+from policyhandler.config import Config
from policyhandler.onap.audit import (REQUEST_X_ECOMP_REQUESTID, Audit,
AuditHttpCode)
-from policyhandler.policy_consts import POLICY_NAME
+from policyhandler.pdp_api_v0.pdp_consts import POLICY_NAME
from policyhandler.policy_receiver import PolicyReceiver
+from policyhandler.utils import Utils
from policyhandler.web_server import _PolicyWeb
-from .mock_policy_engine import MockPolicyEngine
-from .mock_settings import Settings
-from .mock_tracker import Tracker
+from ..mock_settings import MockSettings
+from ..mock_tracker import Tracker
+from .mock_policy_engine import MockPolicyEngine2018
from .mock_websocket import MockWebSocket
+_LOGGER = Utils.get_logger(__file__)
@pytest.mark.usefixtures(
+ "fix_pdp_api_v0",
"fix_discovery",
"fix_pdp_post",
"fix_select_latest_policies_boom"
)
-class WebServerInternalBoomTest(CPWebCase):
+class WebServerInternalBoom2018Test(CPWebCase):
"""testing the web-server - runs tests in alphabetical order of method names"""
def setup_server():
"""setup the web-server"""
@@ -54,21 +56,21 @@
def test_web_healthcheck(self):
"""test /healthcheck"""
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
- Settings.logger.info("got healthcheck: %s", self.body)
+ _LOGGER.info("healthcheck result: %s", result)
+ _LOGGER.info("got healthcheck: %s", self.body)
self.assertStatus('200 OK')
Tracker.validate()
def test_web_policy_latest(self):
"""test /policy_latest/<policy-id>"""
- policy_id, _ = MockPolicyEngine.gen_policy_latest(3)
+ policy_id, _ = MockPolicyEngine2018.gen_policy_latest(3)
self.getPage("/policy_latest/{0}".format(policy_id or ""))
self.assertStatus(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
@@ -76,18 +78,23 @@
def test_web_all_policies_latest(self):
"""test GET /policies_latest"""
result = self.getPage("/policies_latest")
- Settings.logger.info("result: %s", result)
- Settings.logger.info("body: %s", self.body)
+ _LOGGER.info("result: %s", result)
+ _LOGGER.info("body: %s", self.body)
+
+ if Config.is_pdp_api_default():
+ self.assertStatus('404 Not Found')
+ return
+
self.assertStatus(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
def test_web_policies_latest(self):
"""test POST /policies_latest with policyName"""
- match_to_policy_name = MockPolicyEngine.scope_prefix + "amet.*"
+ match_to_policy_name = MockPolicyEngine2018.scope_prefix + "amet.*"
body = json.dumps({POLICY_NAME: match_to_policy_name})
result = self.getPage("/policies_latest", method='POST',
@@ -97,36 +104,45 @@
("Content-Type", "application/json"),
('Content-Length', str(len(body)))
])
- Settings.logger.info("result: %s", result)
- Settings.logger.info("body: %s", self.body)
+ _LOGGER.info("result: %s", result)
+ _LOGGER.info("body: %s", self.body)
+
+ if Config.is_pdp_api_default():
+ self.assertStatus('404 Not Found')
+ return
+
self.assertStatus(AuditHttpCode.SERVER_INTERNAL_ERROR.value)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
Tracker.validate()
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_policy_updates_and_catch_ups(self):
"""test run policy handler with policy updates and catchups"""
- Settings.logger.info("start policy_updates_and_catch_ups")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start policy_updates_and_catch_ups")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_policy_updates_and_catch_ups",
req_message="start policy_updates_and_catch_ups")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1, 3, 5])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.logger.info("sleep 10 before shutdown...")
+ _LOGGER.info("sleep 10 before shutdown...")
time.sleep(10)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -137,31 +153,35 @@
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_catch_up_on_deploy_handler_changed(self):
"""test run policy handler with deployment-handler changed underneath"""
- Settings.logger.info("start zzz_catch_up_on_deploy_handler_changed")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start zzz_catch_up_on_deploy_handler_changed")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_catch_up_on_deploy_handler_changed",
req_message="start zzz_catch_up_on_deploy_handler_changed")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.deploy_handler_instance_uuid = str(uuid.uuid4())
- Settings.logger.info("new deploy-handler uuid=%s", Settings.deploy_handler_instance_uuid)
+ MockSettings.deploy_handler_instance_uuid = str(uuid.uuid4())
+ _LOGGER.info("new deploy-handler uuid=%s", MockSettings.deploy_handler_instance_uuid)
MockWebSocket.send_notification([2, 4])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
- Settings.logger.info("sleep 5 before shutdown...")
+ _LOGGER.info("sleep 5 before shutdown...")
time.sleep(5)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -172,21 +192,25 @@
@pytest.mark.usefixtures("fix_deploy_handler", "fix_policy_receiver_websocket")
def test_zzz_get_catch_up(self):
"""test /catch_up"""
- Settings.logger.info("start /catch_up")
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
+ _LOGGER.info("start /catch_up")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzz_get_catch_up", req_message="start /catch_up")
PolicyReceiver.run(audit)
time.sleep(5)
result = self.getPage("/catch_up")
- Settings.logger.info("catch_up result: %s", result)
+ _LOGGER.info("catch_up result: %s", result)
self.assertStatus('200 OK')
- Settings.logger.info("got catch_up: %s", self.body)
+ _LOGGER.info("got catch_up: %s", self.body)
- Settings.logger.info("sleep 5 before shutdown...")
+ _LOGGER.info("sleep 5 before shutdown...")
time.sleep(5)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
PolicyReceiver.shutdown(audit)
time.sleep(1)
@@ -200,29 +224,33 @@
"fix_cherrypy_engine_exit")
def test_zzzzz_shutdown(self):
"""test shutdown"""
- Settings.logger.info("start shutdown")
+ _LOGGER.info("start shutdown")
assert not PolicyReceiver.is_running()
audit = Audit(job_name="test_zzzzz_shutdown", req_message="start shutdown")
PolicyReceiver.run(audit)
- Settings.logger.info("sleep before send_notification...")
+ _LOGGER.info("sleep before send_notification...")
time.sleep(2)
MockWebSocket.send_notification([1, 3, 5])
- Settings.logger.info("sleep after send_notification...")
+ _LOGGER.info("sleep after send_notification...")
time.sleep(3)
result = self.getPage("/healthcheck")
- Settings.logger.info("healthcheck result: %s", result)
+ _LOGGER.info("healthcheck result: %s", result)
- WebServerInternalBoomTest.do_gc_test = False
- Settings.logger.info("shutdown...")
+ WebServerInternalBoom2018Test.do_gc_test = False
+ _LOGGER.info("shutdown...")
audit.audit_done("shutdown")
result = self.getPage("/shutdown")
- Settings.logger.info("shutdown result: %s", result)
+ _LOGGER.info("shutdown result: %s", result)
self.assertStatus('200 OK')
- Settings.logger.info("got shutdown: %s", self.body)
+ _LOGGER.info("got shutdown: %s", self.body)
time.sleep(1)
assert not PolicyReceiver.is_running()
+ if Config.is_pdp_api_default():
+ _LOGGER.info("passive for new PDP API")
+ return
+
Tracker.validate()
diff --git a/tests/utils/__init__.py b/tests/utils/__init__.py
new file mode 100644
index 0000000..5e2fe60
--- /dev/null
+++ b/tests/utils/__init__.py
@@ -0,0 +1,20 @@
+# ================================================================================
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=========================================================
+#
+
+
+# empty __init__.py so that pytest can add correct path to coverage report, -- per pytest
+# best practice guideline
diff --git a/tests/test_step_timer.py b/tests/utils/test_step_timer.py
similarity index 78%
rename from tests/test_step_timer.py
rename to tests/utils/test_step_timer.py
index d4a0df1..152836d 100644
--- a/tests/test_step_timer.py
+++ b/tests/utils/test_step_timer.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2017-2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2017-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,25 +14,20 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test of the step_timer"""
import json
-import logging
import time
from datetime import datetime
-from policyhandler.config import Config
from policyhandler.step_timer import StepTimer
+from policyhandler.utils import Utils
-Config.init_config()
-
+_LOGGER = Utils.get_logger(__file__)
class MockTimerController(object):
"""testing step_timer"""
- logger = logging.getLogger("policy_handler.unit_test.step_timer")
-
INIT = "init"
NEXT = "next"
STARTED = "started"
@@ -64,25 +59,24 @@
"""timer event"""
self.exe_ts = datetime.utcnow()
self.exe_interval = (self.exe_ts - self.status_ts).total_seconds()
- MockTimerController.logger.info("run on_time[%s] (%s, %s) in %s for %s",
- self.run_counter, json.dumps(args), json.dumps(kwargs),
- self.exe_interval, self.get_status())
+ _LOGGER.info("run on_time[%s] (%s, %s) in %s for %s",
+ self.run_counter, json.dumps(args), json.dumps(kwargs),
+ self.exe_interval, self.get_status())
time.sleep(3)
- MockTimerController.logger.info("done on_time[%s] (%s, %s) in %s for %s",
- self.run_counter, json.dumps(args), json.dumps(kwargs),
- self.exe_interval, self.get_status())
+ _LOGGER.info("done on_time[%s] (%s, %s) in %s for %s",
+ self.run_counter, json.dumps(args), json.dumps(kwargs),
+ self.exe_interval, self.get_status())
def verify_last_event(self):
"""assertions needs to be in the main thread"""
if self.exe_interval is None:
- MockTimerController.logger.info("not executed: %s", self.get_status())
+ _LOGGER.info("not executed: %s", self.get_status())
return
- MockTimerController.logger.info("verify exe %s for %s",
- self.exe_interval, self.get_status())
+ _LOGGER.info("verify exe %s for %s", self.exe_interval, self.get_status())
assert self.exe_interval >= (self.interval - 0.01)
assert self.exe_interval < 2 * self.interval
- MockTimerController.logger.info("success %s", self.get_status())
+ _LOGGER.info("success %s", self.get_status())
def run_timer(self):
"""create and start the step_timer"""
@@ -91,11 +85,7 @@
self.set_status(MockTimerController.NEXT)
return
- self.step_timer = StepTimer(
- self.name, self.interval, MockTimerController.on_time,
- MockTimerController.logger,
- self
- )
+ self.step_timer = StepTimer(self.name, self.interval, MockTimerController.on_time, self)
self.step_timer.start()
self.set_status(MockTimerController.STARTED)
@@ -123,7 +113,7 @@
utcnow = datetime.utcnow()
time_step = (utcnow - self.status_ts).total_seconds()
self.status_ts = utcnow
- MockTimerController.logger.info("[%s]: %s", time_step, self.get_status())
+ _LOGGER.info("[%s]: %s", time_step, self.get_status())
def get_status(self):
"""string representation"""
@@ -138,7 +128,7 @@
def test_step_timer():
"""test step_timer"""
- MockTimerController.logger.info("============ test_step_timer =========")
+ _LOGGER.info("============ test_step_timer =========")
with MockTimerController("step_timer", 5) as step_timer:
step_timer.run_timer()
time.sleep(1)
@@ -166,7 +156,7 @@
def test_interrupt_step_timer():
"""test step_timer"""
- MockTimerController.logger.info("============ test_interrupt_step_timer =========")
+ _LOGGER.info("============ test_interrupt_step_timer =========")
with MockTimerController("step_timer", 5) as step_timer:
step_timer.run_timer()
time.sleep(1)
diff --git a/tests/test_policy_utils.py b/tests/utils/test_utils.py
similarity index 84%
rename from tests/test_policy_utils.py
rename to tests/utils/test_utils.py
index dcf6ccb..18026ff 100644
--- a/tests/test_policy_utils.py
+++ b/tests/utils/test_utils.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,56 +14,51 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test of the policy_utils"""
import json
-import logging
import re
-from policyhandler.config import Config
-from policyhandler.policy_utils import RegexCoarser
+from policyhandler.utils import RegexCoarser, Utils
-Config.init_config()
-LOGGER = logging.getLogger("policy_handler.unit_test_policy_utils")
-
+_LOGGER = Utils.get_logger(__file__)
def check_coarse_regex(test_name, patterns, matching_strings=None, expected_subpatterns=None):
"""generic test"""
regex_coarser = RegexCoarser(patterns)
coarse_patterns = regex_coarser.get_coarse_regex_patterns(max_length=20)
- LOGGER.info("check_coarse_regex %s (%s) for [%s]",
- test_name, coarse_patterns, json.dumps(regex_coarser.patterns))
+ _LOGGER.info("check_coarse_regex %s (%s) for [%s]",
+ test_name, coarse_patterns, json.dumps(regex_coarser.patterns))
coarse_regexes = [re.compile(coarse_pattern) for coarse_pattern in coarse_patterns]
coarse_patterns_str = json.dumps(coarse_patterns)
if matching_strings:
for test_str in matching_strings:
- LOGGER.info(" match '%s' to %s (%s)", test_str, test_name, coarse_patterns_str)
+ _LOGGER.info(" match '%s' to %s (%s)", test_str, test_name, coarse_patterns_str)
assert bool(list(filter(None, [
coarse_regex.match(test_str) for coarse_regex in coarse_regexes
])))
if expected_subpatterns:
for subpattern in expected_subpatterns:
- LOGGER.info(" subpattern '%s' in %s", subpattern, coarse_patterns_str)
+ _LOGGER.info(" subpattern '%s' in %s", subpattern, coarse_patterns_str)
assert subpattern in coarse_patterns_str
def check_combined_regex(test_name, patterns, matching_strings=None, unmatching_strings=None):
"""generic test"""
regex_coarser = RegexCoarser(patterns)
combined_pattern = regex_coarser.get_combined_regex_pattern()
- LOGGER.info("check_combined_regex %s (%s) for [%s]",
- test_name, combined_pattern, json.dumps(regex_coarser.patterns))
+ _LOGGER.info("check_combined_regex %s (%s) for [%s]",
+ test_name, combined_pattern, json.dumps(regex_coarser.patterns))
coarse_regex = re.compile(combined_pattern)
if matching_strings:
for test_str in matching_strings:
- LOGGER.info(" match '%s' to %s (%s)", test_str, test_name, combined_pattern)
+ _LOGGER.info(" match '%s' to %s (%s)", test_str, test_name, combined_pattern)
assert coarse_regex.match(test_str)
if unmatching_strings:
for test_str in unmatching_strings:
- LOGGER.info(" not match '%s' to %s (%s)", test_str, test_name, combined_pattern)
+ _LOGGER.info(" not match '%s' to %s (%s)", test_str, test_name, combined_pattern)
assert not coarse_regex.match(test_str)
def test_regex_coarser():
diff --git a/tests/test_zzz_memory.py b/tests/utils/test_zzz_memory.py
similarity index 83%
rename from tests/test_zzz_memory.py
rename to tests/utils/test_zzz_memory.py
index e1b5af3..4b934cb 100644
--- a/tests/test_zzz_memory.py
+++ b/tests/utils/test_zzz_memory.py
@@ -1,5 +1,5 @@
# ============LICENSE_START=======================================================
-# Copyright (c) 2017-2018 AT&T Intellectual Property. All rights reserved.
+# Copyright (c) 2017-2019 AT&T Intellectual Property. All rights reserved.
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,7 +14,6 @@
# limitations under the License.
# ============LICENSE_END=========================================================
#
-# ECOMP is a trademark and service mark of AT&T Intellectual Property.
"""test of the package for policy-handler of DCAE-Controller"""
@@ -22,12 +21,10 @@
import json
import time
-import pytest
-
from policyhandler.onap.audit import Audit, AuditHttpCode, Metrics
+from policyhandler.utils import Utils
-from .mock_settings import Settings
-
+_LOGGER = Utils.get_logger(__file__)
class Node(object):
"""making the cycled objects"""
@@ -50,7 +47,7 @@
health = audit.health(full=True)
audit.audit_done(result=json.dumps(health))
- Settings.logger.info("healthcheck: %s", json.dumps(health))
+ _LOGGER.info("healthcheck: %s", json.dumps(health))
assert bool(health)
@@ -74,7 +71,7 @@
health = audit.health(full=True)
audit.audit_done(result=json.dumps(health))
- Settings.logger.info("healthcheck: %s", json.dumps(health))
+ _LOGGER.info("healthcheck: %s", json.dumps(health))
assert bool(health)
@@ -97,22 +94,22 @@
health = audit.health(full=True)
audit.audit_done(result=json.dumps(health))
- Settings.logger.info("test_healthcheck_with_garbage[%s]: %s", gc_found, json.dumps(health))
+ _LOGGER.info("test_healthcheck_with_garbage[%s]: %s", gc_found, json.dumps(health))
assert bool(health)
assert bool(health.get("runtime", {}).get("gc", {}).get("gc_garbage"))
- Settings.logger.info("clearing up garbage...")
+ _LOGGER.info("clearing up garbage...")
for obj in gc.garbage:
if isinstance(obj, Node):
- Settings.logger.info("in garbage: %s 0x%x", obj, id(obj))
+ _LOGGER.info("in garbage: %s 0x%x", obj, id(obj))
obj.next = None
gc_found = gc.collect()
- Settings.logger.info("after clear test_healthcheck_with_garbage[%s]: %s",
+ _LOGGER.info("after clear test_healthcheck_with_garbage[%s]: %s",
gc_found, json.dumps(audit.health(full=True)))
gc.set_debug(False)
gc_found = gc.collect()
- Settings.logger.info("after turned off gc debug test_healthcheck_with_garbage[%s]: %s",
+ _LOGGER.info("after turned off gc debug test_healthcheck_with_garbage[%s]: %s",
gc_found, json.dumps(audit.health(full=True)))
diff --git a/version.properties b/version.properties
index 314217b..c5c1fd9 100644
--- a/version.properties
+++ b/version.properties
@@ -1,5 +1,5 @@
-major=4
-minor=6
+major=5
+minor=0
patch=0
base_version=${major}.${minor}.${patch}
release_version=${base_version}