blob: 9c0132cbf85f4fd51f086608f6b223571db42e3f [file] [log] [blame]
Tommy Carpenter42493982019-11-06 07:27:16 -05001.. This work is licensed under a Creative Commons Attribution 4.0 International License.
2.. http://creativecommons.org/licenses/by/4.0
Tommy Carpentercc2250a2019-05-30 15:06:37 -04003
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -04004Developer Guide
5===============
Tommy Carpenter42493982019-11-06 07:27:16 -05006
7.. contents::
8 :depth: 3
9 :local:
Tommy Carpenter5ad8f032019-05-30 14:33:21 -040010
11Tech Stack
Tommy Carpenter42493982019-11-06 07:27:16 -050012----------
Tommy Carpenter5ad8f032019-05-30 14:33:21 -040013
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -040014The A1 Mediator is implemented in Python, currently version 3.8, and
15depends on these third-party packages and technologies:
Lott, Christopher (cl778h)2222dfa2020-05-04 09:39:20 -040016
17- OpenAPI3
18- Connexion
19- Flask with Gevent serving
20- Swagger
Lott, Christopher (cl778h)c91a4a12020-05-27 15:56:20 -040021- Prometheus
22
Tommy Carpenter5ad8f032019-05-30 14:33:21 -040023
Lott, Christopher (cl778h)14d016e2020-04-27 10:45:47 -040024Version bumping A1
25------------------
Tommy Carpenter5ad8f032019-05-30 14:33:21 -040026
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -040027This project follows semver. When the version string changes, these
28files must be updated:
Tommy Carpenter5ad8f032019-05-30 14:33:21 -040029
Lott, Christopher (cl778h)2222dfa2020-05-04 09:39:20 -040030#. ``setup.py``
31#. ``container-tag.yaml``
32#. ``integration_tests/a1mediator/Chart.yaml``
33#. ``docs/release-notes.rst``
34#. ``a1/openapi.yaml`` But note this is an API version, not a software version; there's no need to bump on non-API changes.
35#. And over in the ric-plt/ric-dep repo that contains the A1 Mediator helm chart, files ``values.yaml`` and ``Chart.yaml``.
Tommy Carpenter5ad8f032019-05-30 14:33:21 -040036
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -040037It's convenient to use the Python utility `bumpversion` to maintain
38the first three items. After setup (``pip install bumpversion``) you
39can change the patch version like this::
Tommy Carpenter5ad8f032019-05-30 14:33:21 -040040
Lott, Christopher (cl778h)2222dfa2020-05-04 09:39:20 -040041 bumpversion --verbose patch
Tommy Carpenter810fcb72019-06-05 12:42:52 -040042
Lott, Christopher (cl778h)2222dfa2020-05-04 09:39:20 -040043Or change the minor version like this::
Tommy Carpenter18dcbc32019-06-04 10:20:30 -040044
Lott, Christopher (cl778h)2222dfa2020-05-04 09:39:20 -040045 bumpversion --verbose minor
Tommy Carpenter5ad8f032019-05-30 14:33:21 -040046
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -040047After the `bumpversion` utility has modified the files, update the
48release notes then commit.
Tommy Carpenter971c91c2019-06-12 09:25:48 -040049
Lott, Christopher (cl778h)d5024702019-08-20 15:22:45 -040050
Lott, Christopher (cl778h)14d016e2020-04-27 10:45:47 -040051Version bumping RMR
Tommy Carpenter78ba2732020-02-07 14:06:20 -050052-------------------
Lott, Christopher (cl778h)14d016e2020-04-27 10:45:47 -040053
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -040054A1 (Dockerfile), Dockerfile-Unit-Test, and all three integration test
55receivers use an Alpine base image and install RMR from a base builder
56image. Must update and rebuild all 5 containers in the A1 repo (or
57just A1 itself for production usage).
Tommy Carpenter102b8952020-03-20 10:02:46 -040058
Lott, Christopher (cl778h)d26734f2020-04-27 17:25:05 -040059In addition these items in this repo must be kept in sync:
Lott, Christopher (cl778h)2222dfa2020-05-04 09:39:20 -040060
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -040061#. ``rmr-version.yaml`` controls what rmr gets installed for unit
62 testing in Jenkins
63#. ``integration_tests/install_rmr.sh`` is a useful script for a
64 variety of local testing.
Lott, Christopher (cl778h)d5024702019-08-20 15:22:45 -040065
Lott, Christopher (cl778h)c91a4a12020-05-27 15:56:20 -040066
Lott, Christopher (cl778h)14d016e2020-04-27 10:45:47 -040067Version bumping Python
68----------------------
69
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -040070If you want to update the version of python; for example this was
71recently done to move from 3.7 to 3.8, update these files:
Tommy Carpenter8bcc51a2019-10-21 16:07:31 -040072
Lott, Christopher (cl778h)2222dfa2020-05-04 09:39:20 -040073#. ``Dockerfile``
74#. ``Dockerfile-Unit-Test``
75#. ``tox.ini``
Lott, Christopher (cl778h)d5024702019-08-20 15:22:45 -040076
Lott, Christopher (cl778h)c91a4a12020-05-27 15:56:20 -040077
78Running A1 Standalone
79---------------------
80
81The A1 container can be run standalone, which means using an in-memory mock
82version of SDL and a static route table. The host machine must have the RMR
83library and the environment must define the variable `prometheus_multiproc_dir`
84with a value like /tmp. Alternately, use the following command to run A1 as
85a Docker container, using a route table mounted as a file from this git
86repository and exposing the server's HTTP port on the Docker host::
87
88 docker run -e USE_FAKE_SDL=True -p 10000:10000 -v `pwd`:/opt/route [DOCKER_IMAGE_ID_HERE]
89
90Then test the server with an invocation such as this::
91
92 curl localhost:10000/a1-p/healthcheck
93
94
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -040095Unit Testing
Tommy Carpenter42493982019-11-06 07:27:16 -050096------------
Lott, Christopher (cl778h)14d016e2020-04-27 10:45:47 -040097
Lott, Christopher (cl778h)fe30c172020-04-29 15:23:28 -040098Running the unit tests requires the python packages ``tox`` and ``pytest``.
99
Lott, Christopher (cl778h)39c9ab62020-05-27 09:45:49 -0400100The RMR library is also required during unit tests. If running
101directly from tox (outside a Docker container), install RMR using the
102script in the integration_tests directory: ``install_rmr.sh``.
Lott, Christopher (cl778h)fe30c172020-04-29 15:23:28 -0400103
104Upon completion, view the test coverage like this:
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400105
106::
107
108 tox
109 open htmlcov/index.html
110
Lott, Christopher (cl778h)d26734f2020-04-27 17:25:05 -0400111Alternatively, you can run the unit tests in Docker (this is somewhat
112less nice because you don't get the pretty HTML)
Tommy Carpenter296f8de2019-08-07 11:38:44 -0400113
114::
115
Lott, Christopher (cl778h)fe30c172020-04-29 15:23:28 -0400116 docker build --no-cache -f Dockerfile-Unit-Test .
Tommy Carpenter296f8de2019-08-07 11:38:44 -0400117
Lott, Christopher (cl778h)c91a4a12020-05-27 15:56:20 -0400118
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400119Integration testing
Tommy Carpenter42493982019-11-06 07:27:16 -0500120-------------------
Lott, Christopher (cl778h)14d016e2020-04-27 10:45:47 -0400121
Lott, Christopher (cl778h)d26734f2020-04-27 17:25:05 -0400122This tests A1’s external API with three test receivers. This requires
123docker, kubernetes and helm.
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400124
Lott, Christopher (cl778h)75d92b72020-04-24 11:52:42 -0400125Build all the images:
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400126
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400127::
128
Lott, Christopher (cl778h)75d92b72020-04-24 11:52:42 -0400129 docker build -t a1:latest .
130 cd integration_tests/testxappcode
131 docker build -t delayreceiver:latest -f Dockerfile-delay-receiver .
132 docker build -t queryreceiver:latest -f Dockerfile-query-receiver .
133 docker build -t testreceiver:latest -f Dockerfile-test-receiver .
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400134
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400135
Tommy Carpenter9d5ad712019-12-02 11:02:01 -0500136Then, run all the tests from the root (this requires the python packages ``tox``, ``pytest``, and ``tavern``).
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400137
Tommy Carpenter7dbbfbc2019-06-18 09:58:55 -0400138::
139
140 tox -c tox-integration.ini
141
142This script:
Lott, Christopher (cl778h)2222dfa2020-05-04 09:39:20 -0400143
144#. Deploys 3 helm charts (5 containers) into a local kubernetes installation
145#. Port forwards a pod ClusterIP to localhost
146#. Uses “tavern” to run some tests against the server
147#. Barrages the server with Apache bench
148#. Tears everything down