Tommy Carpenter | 4249398 | 2019-11-06 07:27:16 -0500 | [diff] [blame] | 1 | .. This work is licensed under a Creative Commons Attribution 4.0 International License. |
| 2 | .. http://creativecommons.org/licenses/by/4.0 |
Tommy Carpenter | cc2250a | 2019-05-30 15:06:37 -0400 | [diff] [blame] | 3 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 4 | Developer Guide |
| 5 | =============== |
Tommy Carpenter | 4249398 | 2019-11-06 07:27:16 -0500 | [diff] [blame] | 6 | |
| 7 | .. contents:: |
| 8 | :depth: 3 |
| 9 | :local: |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 10 | |
| 11 | Tech Stack |
Tommy Carpenter | 4249398 | 2019-11-06 07:27:16 -0500 | [diff] [blame] | 12 | ---------- |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 13 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 14 | The A1 Mediator is implemented in Python, currently version 3.8, and |
| 15 | depends on these third-party packages and technologies: |
Lott, Christopher (cl778h) | 2222dfa | 2020-05-04 09:39:20 -0400 | [diff] [blame] | 16 | |
| 17 | - OpenAPI3 |
| 18 | - Connexion |
| 19 | - Flask with Gevent serving |
| 20 | - Swagger |
Lott, Christopher (cl778h) | c91a4a1 | 2020-05-27 15:56:20 -0400 | [diff] [blame] | 21 | - Prometheus |
| 22 | |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 23 | |
Lott, Christopher (cl778h) | 14d016e | 2020-04-27 10:45:47 -0400 | [diff] [blame] | 24 | Version bumping A1 |
| 25 | ------------------ |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 26 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 27 | This project follows semver. When the version string changes, these |
| 28 | files must be updated: |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 29 | |
Lott, Christopher (cl778h) | 2222dfa | 2020-05-04 09:39:20 -0400 | [diff] [blame] | 30 | #. ``setup.py`` |
| 31 | #. ``container-tag.yaml`` |
| 32 | #. ``integration_tests/a1mediator/Chart.yaml`` |
| 33 | #. ``docs/release-notes.rst`` |
| 34 | #. ``a1/openapi.yaml`` But note this is an API version, not a software version; there's no need to bump on non-API changes. |
| 35 | #. And over in the ric-plt/ric-dep repo that contains the A1 Mediator helm chart, files ``values.yaml`` and ``Chart.yaml``. |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 36 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 37 | It's convenient to use the Python utility `bumpversion` to maintain |
| 38 | the first three items. After setup (``pip install bumpversion``) you |
| 39 | can change the patch version like this:: |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 40 | |
Lott, Christopher (cl778h) | 2222dfa | 2020-05-04 09:39:20 -0400 | [diff] [blame] | 41 | bumpversion --verbose patch |
Tommy Carpenter | 810fcb7 | 2019-06-05 12:42:52 -0400 | [diff] [blame] | 42 | |
Lott, Christopher (cl778h) | 2222dfa | 2020-05-04 09:39:20 -0400 | [diff] [blame] | 43 | Or change the minor version like this:: |
Tommy Carpenter | 18dcbc3 | 2019-06-04 10:20:30 -0400 | [diff] [blame] | 44 | |
Lott, Christopher (cl778h) | 2222dfa | 2020-05-04 09:39:20 -0400 | [diff] [blame] | 45 | bumpversion --verbose minor |
Tommy Carpenter | 5ad8f03 | 2019-05-30 14:33:21 -0400 | [diff] [blame] | 46 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 47 | After the `bumpversion` utility has modified the files, update the |
| 48 | release notes then commit. |
Tommy Carpenter | 971c91c | 2019-06-12 09:25:48 -0400 | [diff] [blame] | 49 | |
Lott, Christopher (cl778h) | d502470 | 2019-08-20 15:22:45 -0400 | [diff] [blame] | 50 | |
Lott, Christopher (cl778h) | 14d016e | 2020-04-27 10:45:47 -0400 | [diff] [blame] | 51 | Version bumping RMR |
Tommy Carpenter | 78ba273 | 2020-02-07 14:06:20 -0500 | [diff] [blame] | 52 | ------------------- |
Lott, Christopher (cl778h) | 14d016e | 2020-04-27 10:45:47 -0400 | [diff] [blame] | 53 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 54 | A1 (Dockerfile), Dockerfile-Unit-Test, and all three integration test |
| 55 | receivers use an Alpine base image and install RMR from a base builder |
| 56 | image. Must update and rebuild all 5 containers in the A1 repo (or |
| 57 | just A1 itself for production usage). |
Tommy Carpenter | 102b895 | 2020-03-20 10:02:46 -0400 | [diff] [blame] | 58 | |
Lott, Christopher (cl778h) | d26734f | 2020-04-27 17:25:05 -0400 | [diff] [blame] | 59 | In addition these items in this repo must be kept in sync: |
Lott, Christopher (cl778h) | 2222dfa | 2020-05-04 09:39:20 -0400 | [diff] [blame] | 60 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 61 | #. ``rmr-version.yaml`` controls what rmr gets installed for unit |
| 62 | testing in Jenkins |
| 63 | #. ``integration_tests/install_rmr.sh`` is a useful script for a |
| 64 | variety of local testing. |
Lott, Christopher (cl778h) | d502470 | 2019-08-20 15:22:45 -0400 | [diff] [blame] | 65 | |
Lott, Christopher (cl778h) | c91a4a1 | 2020-05-27 15:56:20 -0400 | [diff] [blame] | 66 | |
Lott, Christopher (cl778h) | 14d016e | 2020-04-27 10:45:47 -0400 | [diff] [blame] | 67 | Version bumping Python |
| 68 | ---------------------- |
| 69 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 70 | If you want to update the version of python; for example this was |
| 71 | recently done to move from 3.7 to 3.8, update these files: |
Tommy Carpenter | 8bcc51a | 2019-10-21 16:07:31 -0400 | [diff] [blame] | 72 | |
Lott, Christopher (cl778h) | 2222dfa | 2020-05-04 09:39:20 -0400 | [diff] [blame] | 73 | #. ``Dockerfile`` |
| 74 | #. ``Dockerfile-Unit-Test`` |
| 75 | #. ``tox.ini`` |
Lott, Christopher (cl778h) | d502470 | 2019-08-20 15:22:45 -0400 | [diff] [blame] | 76 | |
Lott, Christopher (cl778h) | c91a4a1 | 2020-05-27 15:56:20 -0400 | [diff] [blame] | 77 | |
| 78 | Running A1 Standalone |
| 79 | --------------------- |
| 80 | |
| 81 | The A1 container can be run standalone, which means using an in-memory mock |
| 82 | version of SDL and a static route table. The host machine must have the RMR |
| 83 | library and the environment must define the variable `prometheus_multiproc_dir` |
| 84 | with a value like /tmp. Alternately, use the following command to run A1 as |
| 85 | a Docker container, using a route table mounted as a file from this git |
| 86 | repository and exposing the server's HTTP port on the Docker host:: |
| 87 | |
| 88 | docker run -e USE_FAKE_SDL=True -p 10000:10000 -v `pwd`:/opt/route [DOCKER_IMAGE_ID_HERE] |
| 89 | |
| 90 | Then test the server with an invocation such as this:: |
| 91 | |
| 92 | curl localhost:10000/a1-p/healthcheck |
| 93 | |
| 94 | |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 95 | Unit Testing |
Tommy Carpenter | 4249398 | 2019-11-06 07:27:16 -0500 | [diff] [blame] | 96 | ------------ |
Lott, Christopher (cl778h) | 14d016e | 2020-04-27 10:45:47 -0400 | [diff] [blame] | 97 | |
Lott, Christopher (cl778h) | fe30c17 | 2020-04-29 15:23:28 -0400 | [diff] [blame] | 98 | Running the unit tests requires the python packages ``tox`` and ``pytest``. |
| 99 | |
Lott, Christopher (cl778h) | 39c9ab6 | 2020-05-27 09:45:49 -0400 | [diff] [blame] | 100 | The RMR library is also required during unit tests. If running |
| 101 | directly from tox (outside a Docker container), install RMR using the |
| 102 | script in the integration_tests directory: ``install_rmr.sh``. |
Lott, Christopher (cl778h) | fe30c17 | 2020-04-29 15:23:28 -0400 | [diff] [blame] | 103 | |
| 104 | Upon completion, view the test coverage like this: |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 105 | |
| 106 | :: |
| 107 | |
| 108 | tox |
| 109 | open htmlcov/index.html |
| 110 | |
Lott, Christopher (cl778h) | d26734f | 2020-04-27 17:25:05 -0400 | [diff] [blame] | 111 | Alternatively, you can run the unit tests in Docker (this is somewhat |
| 112 | less nice because you don't get the pretty HTML) |
Tommy Carpenter | 296f8de | 2019-08-07 11:38:44 -0400 | [diff] [blame] | 113 | |
| 114 | :: |
| 115 | |
Lott, Christopher (cl778h) | fe30c17 | 2020-04-29 15:23:28 -0400 | [diff] [blame] | 116 | docker build --no-cache -f Dockerfile-Unit-Test . |
Tommy Carpenter | 296f8de | 2019-08-07 11:38:44 -0400 | [diff] [blame] | 117 | |
Lott, Christopher (cl778h) | c91a4a1 | 2020-05-27 15:56:20 -0400 | [diff] [blame] | 118 | |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 119 | Integration testing |
Tommy Carpenter | 4249398 | 2019-11-06 07:27:16 -0500 | [diff] [blame] | 120 | ------------------- |
Lott, Christopher (cl778h) | 14d016e | 2020-04-27 10:45:47 -0400 | [diff] [blame] | 121 | |
Lott, Christopher (cl778h) | d26734f | 2020-04-27 17:25:05 -0400 | [diff] [blame] | 122 | This tests A1’s external API with three test receivers. This requires |
| 123 | docker, kubernetes and helm. |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 124 | |
Lott, Christopher (cl778h) | 75d92b7 | 2020-04-24 11:52:42 -0400 | [diff] [blame] | 125 | Build all the images: |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 126 | |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 127 | :: |
| 128 | |
Lott, Christopher (cl778h) | 75d92b7 | 2020-04-24 11:52:42 -0400 | [diff] [blame] | 129 | docker build -t a1:latest . |
| 130 | cd integration_tests/testxappcode |
| 131 | docker build -t delayreceiver:latest -f Dockerfile-delay-receiver . |
| 132 | docker build -t queryreceiver:latest -f Dockerfile-query-receiver . |
| 133 | docker build -t testreceiver:latest -f Dockerfile-test-receiver . |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 134 | |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 135 | |
Tommy Carpenter | 9d5ad71 | 2019-12-02 11:02:01 -0500 | [diff] [blame] | 136 | Then, run all the tests from the root (this requires the python packages ``tox``, ``pytest``, and ``tavern``). |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 137 | |
Tommy Carpenter | 7dbbfbc | 2019-06-18 09:58:55 -0400 | [diff] [blame] | 138 | :: |
| 139 | |
| 140 | tox -c tox-integration.ini |
| 141 | |
| 142 | This script: |
Lott, Christopher (cl778h) | 2222dfa | 2020-05-04 09:39:20 -0400 | [diff] [blame] | 143 | |
| 144 | #. Deploys 3 helm charts (5 containers) into a local kubernetes installation |
| 145 | #. Port forwards a pod ClusterIP to localhost |
| 146 | #. Uses “tavern” to run some tests against the server |
| 147 | #. Barrages the server with Apache bench |
| 148 | #. Tears everything down |