Tommy Carpenter | cc2250a | 2019-05-30 15:06:37 -0400 | [diff] [blame] | 1 | .. ================================================================================== |
| 2 | .. Copyright (c) 2019 Nokia |
| 3 | .. Copyright (c) 2018-2019 AT&T Intellectual Property. |
| 4 | .. |
| 5 | .. Licensed under the Apache License, Version 2.0 (the "License"); |
| 6 | .. you may not use this file except in compliance with the License. |
| 7 | .. You may obtain a copy of the License at |
| 8 | .. |
| 9 | .. http://www.apache.org/licenses/LICENSE-2.0 |
| 10 | .. |
| 11 | .. Unless required by applicable law or agreed to in writing, software |
| 12 | .. distributed under the License is distributed on an "AS IS" BASIS, |
| 13 | .. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 14 | .. See the License for the specific language governing permissions and |
| 15 | .. limitations under the License. |
| 16 | .. ================================================================================== |
| 17 | |
| 18 | A1 Dev Guide |
| 19 | ============ |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 20 | |
| 21 | Tech Stack |
| 22 | ========== |
| 23 | |
| 24 | - OpenAPI3 |
| 25 | - Connexion |
| 26 | - Flask with Gevent serving |
| 27 | - Python3.7 |
| 28 | |
| 29 | Version bumping |
| 30 | =============== |
| 31 | |
| 32 | This project follows semver. When changes are made, the versions are in: |
| 33 | |
| 34 | 1) ``docs/release-notes.rst`` |
| 35 | |
| 36 | 2) ``setup.py`` |
| 37 | |
Tommy Carpenter | 810fcb7 | 2019-06-05 12:42:52 -0400 | [diff] [blame] | 38 | 3) ``container-tag.yaml`` |
| 39 | |
Lott, Christopher (cl778h) | 438a0a6 | 2019-06-18 06:50:13 -0400 | [diff] [blame] | 40 | 4) ``integration_tests/a1mediator/Chart.yaml`` |
Tommy Carpenter | 18dcbc3 | 2019-06-04 10:20:30 -0400 | [diff] [blame] | 41 | |
Lott, Christopher (cl778h) | 438a0a6 | 2019-06-18 06:50:13 -0400 | [diff] [blame] | 42 | 6) ``a1/openapi.yml`` (this is an API version, not a software version) |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 43 | |
Tommy Carpenter | 971c91c | 2019-06-12 09:25:48 -0400 | [diff] [blame] | 44 | 7) in the it/dep repo that contains a1 helm chart, ``values.yaml``, ``Chart.yml`` |
| 45 | |
Lott, Christopher (cl778h) | d502470 | 2019-08-20 15:22:45 -0400 | [diff] [blame] | 46 | Version bumping rmr-python |
| 47 | ========================== |
| 48 | rmr-python is a critial dependency of A1. Bumping the rmr version dependency requires changes in: |
| 49 | |
| 50 | 1) ``setup.py`` |
| 51 | |
| 52 | 2) ``Dockerfile`` |
| 53 | |
| 54 | 3) ``integration_tests/Dockerfile`` |
| 55 | |
| 56 | Run the integration tests after attempting this. |
| 57 | |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 58 | Unit Testing |
| 59 | ============ |
Tommy Carpenter | 2451446 | 2019-07-16 11:25:52 -0400 | [diff] [blame] | 60 | Note, before this will work, for the first time on the machine running the tests, run ``./install_deps.sh``. This is only needed once on the machine. |
| 61 | Also, this requires the python packages ``tox`` and ``pytest``. |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 62 | |
| 63 | :: |
| 64 | |
| 65 | tox |
| 66 | open htmlcov/index.html |
| 67 | |
Tommy Carpenter | 296f8de | 2019-08-07 11:38:44 -0400 | [diff] [blame] | 68 | Alternatively, you can run the unit tests in Docker (this is somewhat less nice because you don't get the pretty HTML) |
| 69 | |
| 70 | :: |
| 71 | |
| 72 | docker build --no-cache -t a1test:latest -f Dockerfile-Unit-Test |
| 73 | |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 74 | Integration testing |
| 75 | =================== |
| 76 | This tests A1’s external API with two test receivers. This depends on helm+k8s, meaning you cannot run this if this is not installed. |
| 77 | |
| 78 | Unlike the unit tests, however, this does not require rmr to be installed on the base system, as everything |
| 79 | runs in Docker, and the Dockerfiles provide/install rmr. |
| 80 | |
| 81 | First, build the latest A1 you are testing (from the root): |
| 82 | :: |
| 83 | |
| 84 | docker build --no-cache -t a1:latest . |
| 85 | |
| 86 | Note that this step also runs the unit tests, since running the unit tests are part of the Dockerfile for A1. |
| 87 | |
| 88 | If you've never run the integration tests before, build the test receiver, which is referenced in the helm chart: |
| 89 | :: |
| 90 | |
| 91 | cd integration_tests |
| 92 | docker build --no-cache -t testreceiver:latest . |
| 93 | |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 94 | Finally, run all the tests from the root (this requires the python packages ``tox``, ``pytest``, and ``tavern``). |
| 95 | :: |
| 96 | |
| 97 | tox -c tox-integration.ini |
| 98 | |
| 99 | This script: |
| 100 | 1. Deploys 3 helm charts into a local kubernetes installation |
| 101 | 2. Port forwards a pod ClusterIP to localhost |
| 102 | 3. Uses “tavern” to run some tests against the server |
| 103 | 4. Barrages the server with apache bench |
| 104 | 5. Tears everything down |
| 105 | |
Tommy Carpenter | fdf0504 | 2019-07-18 20:21:21 +0000 | [diff] [blame] | 106 | Unless you're a core A1 developer, you should probably stop here. The below instructions |
| 107 | are for running A1 locally, without docker, and is much more involved (however useful when developing a1). |
| 108 | |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 109 | Running locally |
| 110 | =============== |
| 111 | |
Tommy Carpenter | 2451446 | 2019-07-16 11:25:52 -0400 | [diff] [blame] | 112 | 1. Before this will work, for the first time on that machine, run ``./install_deps.sh`` |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 113 | |
Lott, Christopher (cl778h) | d502470 | 2019-08-20 15:22:45 -0400 | [diff] [blame] | 114 | 2. It also requires rmr-python installed. (The dockerfile does this) |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 115 | |
| 116 | 3. Create a ``local.rt`` file and copy it into ``/opt/route/local.rt``. |
Tommy Carpenter | fdf0504 | 2019-07-18 20:21:21 +0000 | [diff] [blame] | 117 | Note, the example one in ``integration_tests`` will need to be modified for |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 118 | your scenario and machine. |
| 119 | |
| 120 | 4. Copy a ric manifest into ``/opt/ricmanifest.json`` and an rmr mapping |
| 121 | table into ``/opt/rmr_string_int_mapping.txt``. You can use the test |
| 122 | ones packaged if you want: |
| 123 | |
| 124 | :: |
| 125 | |
Tommy Carpenter | fdf0504 | 2019-07-18 20:21:21 +0000 | [diff] [blame] | 126 | cp tests/fixtures/ricmanifest.json /opt/ricmanifest.json |
| 127 | cp tests/fixtures/rmr_string_int_mapping.txt /opt/rmr_string_int_mapping.txt |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 128 | |
| 129 | 5. Then: |
| 130 | |
Tommy Carpenter | fdf0504 | 2019-07-18 20:21:21 +0000 | [diff] [blame] | 131 | :: |
| 132 | |
| 133 | sudo pip install -e . |
| 134 | set -x LD_LIBRARY_PATH /usr/local/lib/; set -x RMR_SEED_RT /opt/route/local.rt ; set -x RMR_RCV_RETRY_INTERVAL 500; set -x RMR_RETRY_TIMES 10; |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 135 | /usr/bin/run.py |
| 136 | |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 137 | |
Tommy Carpenter | 09edf28 | 2019-06-11 09:10:23 -0400 | [diff] [blame] | 138 | There are also two test receivers in ``integration_tests`` you can run locally. |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 139 | The first is meant to be used with the ``control_admission`` policy |
| 140 | (that comes in test fixture ric manifest): |
| 141 | |
| 142 | :: |
| 143 | |
| 144 | set -x LD_LIBRARY_PATH /usr/local/lib/; set -x RMR_SEED_RT /opt/route/local.rt ; python receiver.py |
| 145 | |
| 146 | The second can be used against the ``test_policy`` policy to test the |
| 147 | async nature of A1, and to test race conditions. You can start it with |
| 148 | several env variables as follows: |
| 149 | |
| 150 | :: |
| 151 | |
| 152 | set -x LD_LIBRARY_PATH /usr/local/lib/; set -x RMR_SEED_RT /opt/route/local.rt ; set -x TEST_RCV_PORT 4563; set -x TEST_RCV_RETURN_MINT 10001; set -x TEST_RCV_SEC_DELAY 5; set -x TEST_RCV_RETURN_PAYLOAD '{"ACK_FROM": "DELAYED_TEST", "status": "SUCCESS"}' ; python receiver.py |
| 153 | |
| 154 | To test the async nature of A1, trigger a call to ``test_policy``, which |
| 155 | will target the delayed receicer, then immediately call |
| 156 | ``control_admission``. The ``control_admission`` policy return should be |
| 157 | returned immediately, whereas the ``test_policy`` should return after |
| 158 | about ``TEST_RCV_SEC_DELAY 5``. The ``test_policy`` should not block A1 |
| 159 | while it is sleeping, and both responses should be correct. |
| 160 | |
| 161 | :: |
| 162 | |
| 163 | curl -v -X PUT -H "Content-Type: application/json" -d '{}' localhost:10000/ric/policies/test_policy |
Tommy Carpenter | 2451446 | 2019-07-16 11:25:52 -0400 | [diff] [blame] | 164 | curl -v -X PUT -H "Content-Type: application/json" -d '{ "enforce":true, "window_length":10, "blocking_rate":20, "trigger_threshold":10 }' localhost:10000/ric/policies/admission_control_policy |
Tommy Carpenter | fdf0504 | 2019-07-18 20:21:21 +0000 | [diff] [blame] | 165 | curl -v localhost:10000/ric/policies/admission_control_policy |
| 166 | curl -v localhost:10000/a1-p/healthcheck |